Media-editing application with a free-form space for organizing or compositing media clips

ABSTRACT

For a media-editing application, some embodiments of the invention provide a novel two-dimensional free-form space that allows a user to both visually organize media clips and composite media clips in order to create composite presentations. To visually organize media clips in the space, some embodiments allow the user to add, remove, visually arrange and rearrange, and minimize the media clips. Also, the media clips in the space can be placed at any location within the entire space when the user is visually organizing the media clips. To composite media clips in the free-form space, some embodiments also allow the user to create sequences of media clips, reorder the media clips within sequences, stack sequences, and add or remove media clips from sequences. In some embodiments, the media clips or created sequences can be placed at any location within the entire space when the user is organizing and compositing media clips.

This application is related to the following application: U.S. patentapplication Ser. No. 12/837,490, filed Jul. 15, 2010, now published asU.S. Patent Publication 2012/0017153.

BACKGROUND

Digital graphic design, image editing, audio editing, and video editingapplications (hereafter collectively referred to as media contentediting applications or media-editing applications) provide graphicaldesigners, media artists, and other users with the necessary tools tocreate a variety of media content. Examples of such applications includeFinal Cut Pro® and iMovie®, both sold by Apple Inc. These applicationsgive users the ability to edit, combine, transition, overlay, and piecetogether different media content in a variety of manners to create aresulting media project. The resulting media project specifies aparticular sequenced composition of any number of text, audio, image,and/or video content elements that is used to create a mediapresentation.

Various media-editing applications facilitate such composition throughelectronic means. Specifically, a computer or other electronic devicewith a processor and computer readable storage medium executes the mediacontent editing application. In so doing, the computer generates agraphical interface whereby designers digitally manipulate graphicalrepresentations of the media content to produce a desired result.

One difficulty in media-editing is that a user cannot easily try outdifferent ideas of compositing media content elements and see theresulting media presentations. For example, to create a mediapresentation, contents in the form of media clips need to be broughtinto a media project, which usually has a timeline and multiple tracks.The timeline and multiple tracks necessarily creates positionalrelationships between the media clips placed in the tracks.Understanding the meaning or effect of placing a media clip on aparticular track at a particular position of the timeline requiresfamiliarity and the sophistication associated with the complexapplications and tends to overwhelm or discourage untrained users.Moreover, there is a sense of commitment or structure in a timeline thatdoes not encourage experimentation or “playing around.”

BRIEF SUMMARY

For a media-editing application that creates composite presentations,some embodiments of the invention provide a novel two-dimensionalfree-form space (e.g., display area) that allows a user to both visuallyorganize media clips and composite media clips in order to createcomposite presentations. To visually organize media clips in thefree-form space, some embodiments allow the user to add, remove,visually arrange and rearrange, and minimize the media clips. Also, themedia clips in the space can be placed at any location within the entirespace when the user is visually organizing the media clips. To compositemedia clips in the free-form space, some embodiments also allow the userto create sequences of media clips, reorder the media clips withinsequences, stack sequences, and add or remove media clips fromsequences. In some embodiments, the media clips or created sequences ofsuch media clips can be placed at any location within the entire spacewhen the user is organizing and compositing media clips.

Some embodiments provide a free-form space that allows the user to placemedia clips (and sequences of media clips) in regions within the spacethat overlap each other at the same or different instances in time. Insome embodiments, a free-form space does not limit the placement ofmedia clips within the free-form space. That is, the media clips can beplaced at any location within the free-form space of such embodiments,as mentioned above.

In some embodiments, the free-form space is a trackless space thatallows the user to place media clips and sequences of media clipswithout being constrained by such tracks (e.g., the media clips andsequence of media clips do not have to be aligned along tracks). Inother words, the free-from space in some embodiments does not includetracks, such as those included in a typical timeline-based compositingarea of a media-editing application that is for compositing mediapresentations. In typical media editing applications, adjacent tracksare non-overlapping regions in which media clips can be placed withoutthe media clips overlapping each other. As mentioned above, thefree-form space of some embodiments media clips and sequences to beplaced in regions that overlap at the same or different instances intime.

Rather than use tracks, the free-form space of some embodiments is agridded space that includes gridlines on which the media clips align.Some embodiments define the gridlines of the space as fine as the pixelsof a display device displaying the space. In such embodiments, thefree-form space is referred to as gridless display area since a mediaclip placed in any position in the free-form space aligns with the pixelgridlines. Other embodiments, however, define the gridlines of the spacemore coarse.

For the space of some embodiments, the positional relation between twoitems in the space (e.g., between two clips, between a sequence and aclip, between two sequences) does not necessarily define anyrelationship between the two items. For instance, because a timelinedoes not span along this space in some embodiments, the positionalrelationship between two items in this space does not necessarily definea temporal relationship between them. As another example, in someembodiments, the positional relationship between two items does notnecessarily define a logical relationship between the two items.However, for some items in the space, the positional relationship doesdefine a relationship when two items have been somehow associated witheach other (e.g., through user action). For example, as described above,a user can create a sequence of media clips (e.g., by concatenatetogether several media clips). In such a concatenated sequence, thesemedia clips do have a temporal positional relationship with each other.Moreover, when two sequences are concatenated or composited together inthis space, the sequences (and the media clips in the sequences) have atemporal positional relationship with each other.

The embodiments described above mention numerous features of a free-formspace. However, other embodiments do not have to include all thefeatures mentioned above. For instance, although the free-form space isa trackless space in some embodiments, this space is divided intoseveral tracks in other embodiments in order to allow the user to bothvisually organize and composite media clips. Furthermore, in someembodiments, the free-form space is one of many spaces through which auser of the media-editing application organizes media clips. Otherspaces can be used to organize media clips as well. In otherembodiments, the free-form space is one of many spaces through which auser of the media-editing application composites media clips. Otherspaces, such as a timeline space, can be used to composite media clips.However, in some embodiments, the free-form space may not include allthe features and functionalities provided by the other spaces while inother embodiments the free-form space does include all the features andfunctionalities provided by the other spaces.

For some embodiments of invention, a graphical user interface (“GUI”) ofa media-editing application includes an organizing and editing free-formspace and a media library through which the application's user canselect media clips to add to the free-form space to organize and edit.The media clips may be displayed with different appearances ordimensions in different areas of the GUI. For instance, the media clipsin the free-form space are represented using thumbnails (i.e., a set ofequally-sized images) while the media clips in the media library arerepresented using smaller thumbnails (i.e., a set of equally-sized, butsmaller, images). As such, different embodiments define the appearanceand dimensions of the media clips differently.

Additional media clips can be added to the free-form space from themedia library at any time. These clips can be added to the free-formspace in several different ways. For instance, in some embodiments, theuser can add a media clip by first selecting a media clip. The selectedmedia clip is then added to the free-form space through a drag-and-dropoperation (i.e., by using a cursor-control device, such as a mouse ortrackpad or a finger for a touch-screen device) to select this mediaclip from the media library and dragging to the free-form space. Asanother example, a media clip can be added to the free-form space byselecting an option from a pull-down menu or a pop-up menu afterselecting a media clip in the media library.

In the free-form space, media clips can be placed near each other toform visual grouping. In some embodiments, media clips in a visualgrouping can be rearranged so that the user can change the visualarrangement of the media clips in the visual grouping. These groupingscan then be rearranged to form different visual groupings. For instance,the user can move one media clip in one visual grouping near media clipsin another visual grouping to form a new visual grouping that includesall of those media clips. Thus, the user can visually group the mediaclips into any number of different visual groupings that the userdesires.

In some embodiments, two media clips closely placed next to each otherautomatically slide away from each other to make room for a third mediaclip when the third media clip is placed between them. In otherembodiments, the two media clips do not move when the third media clipis placed between the two media clips. As such, the third media clip canbe made to overlap the two other media clips.

In addition to visual groupings, the media clips in the free-form spacecan be grouped into logical groupings. For instance, in someembodiments, the user can select several media clips in the free-formspace and create a logical grouping that includes the selected mediaclips by executing a key command (e.g., control-G), a combination of keycommands, a hotkey, or any other appropriate method. Some embodimentsrequire that media clips to be within a particular distance of eachother before they can be logically grouped while other embodiments allowmedia clips located anywhere in the free-form space to be logicallygrouped together. Once a logical grouping of media clip is created, thelogical grouping can be moved within the space as a single item. Forinstance, performing a drag-and-drop operation on one of the logicallygrouped media clips also moves the other media clips in the logicalgroup in the same relative direction and distance as the medial clip onwhich drag-and-drop operation is performed.

In some embodiments, media clips can be minimized or collapsed into asmaller thumbnail or an icon in the free-form space. A media clip can beminimized or collapsed in several different ways. For instance, a usercan minimize or collapse a media clip by selecting the media clip andthen selecting a minimize button that is displayed at a location in theGUI. In some embodiments, the user selects a menu item that is displayedin a pull-down menu that is accessed by selecting a top-level menu inthe GUI or a pop-up menu that is accessed through a cursor controloperation (e.g., a right-click cursor operation or a control-clickcursor operation).

Some embodiments allow a user to play back media clips in the free-formspace. A media clip is played back through a set of playback controls orthrough a menu item selection. A set of playback controls in someembodiments includes a user selectable UI item for backward playback ofa media clip, a user selectable UI item for pausing playback of themedia clip, and a user selectable UI item for forward playback of themedia clip.

In some embodiments, the playback of the selected media clip isdisplayed in the media clip itself while in other embodiments playbackof the selected media clip is displayed in a viewing area which is anarea separate from the free-form space. In yet other embodiments, theplayback of the selected media clip is displayed both in the media clipitself and a viewing area.

Some embodiments display a playhead (or scrubber bar) over a selectedmedia clip that moves horizontally along the selected media clip as itis playing back. The position of the playhead along the displayed lengthof the media clip indicates the position of the content being played.For example, if the media clip is a video clip that includes a sequenceof frames, the position of the playhead along the length of the mediaclip represents the position of the currently displayed frame in thesequence of frames of the video clip.

In some embodiments, the rate at which the playhead moves horizontallyalong the selected media clip being played is different for media clipsthat have different durations. This is because, in some embodiments, thedisplayed length of a media clip represents the duration of the contentassociated with the media clip with the left edge representing thestarting point and the right edge representing the ending point of thecontent and the media clips displayed in the free-form space areuniform-sized rectangles. For example, in such embodiments, a playheadmoves horizontally along a three-minute audio clip faster than aplayhead along a five-minute video clip since the duration of the audioclip is shorter than the duration of the video clip.

In some embodiments, the viewing area is located within the free-formspace while in other embodiments the viewing area is located outside thefree-form space. In some embodiments, the viewing area is shared withother workspaces of the media-editing application. In other words, themedia clips in other workspaces are also played back in the viewingarea. Furthermore, the viewing area of some embodiments is expanded(e.g., by using a keystroke, a combination of keystrokes, a hotkey, apull-down or pop-up menu selection etc.) to occupy the entire monitor ordisplay device (i.e., full screen playback) of the computing device onwhich the media-editing application is being executed.

In some embodiments, a media clip in the free-form space is played backby the user's selecting a play option (e.g., by left-clicking) displayedin a pop-up menu. In some embodiments, the pop-up menu is accessedthrough a cursor control operation (e.g., a right-click cursor operationor a control-click cursor operation). In some embodiments, playback ofthe media clip is only displayed in the viewing area (i.e., playback isnot displayed in the selected media clip itself).

As mentioned above, the free-form space in some embodiments allows auser to create sequences of media clips, reorder the media clips withinsequences, and add or remove media clips from sequences in addition toallowing the user to add, organize, visually arrange, collapse, and playback the media clips. In some embodiments, a sequence of media clips isa chronological grouping of two or more media clips. Some embodimentsform a sequence of media clips by horizontally concatenating media clipsin the free-form space. For example, a sequence that includes a firstmedia clip and a second media clip is formed when the second media clipis placed or moved in the free-form space such that the left edge of thesecond media clip coincides with the right edge of the first media clip.When playing back the sequence, the first media clip is first playedback and then the second media clip is played back. In some embodiments,when a media clip is moved within a threshold distance of another mediaclip in the free-form space, the media clip being moved “snaps” to theother media clip in order to form a sequence.

Some embodiments allow the user to reorder the media clips in a sequencein order to change the chronological order of the media clips in thesequence. For example, in a sequence of three media clips, the thirdmedia clip can be placed between the first two media clips in thesequence. In some embodiments, when a media clip is placed between twomedia clips in a sequence, the two media clips automatically slidehorizontally away from each other in order to make a room for the mediaclip being placed between the two media clips.

Some embodiments allow the user to add a media clip to a sequence ofmedia clips. For example, a media clip which is not part of any sequencecan being added into a sequence of three media clips. To add the mediaclip to this sequence, the user selects the media clip and moves itbetween any two media clips in the sequence (e.g., by performing adrag-and-drop operation) in the sequence. The two media clipsautomatically slide horizontally away from each other in order to make aroom for the media clip to be added.

Some embodiments allow the user to layer media content of a media clipor a sequence of media clips over media content of another media clip orsequence of media clips in a composite presentation composed of thesemedia clips or sequences of media clips. In some such embodiments, mediacontent in one layer (e.g., the top layer) is played back over any mediacontent in the other layer (e.g., the lower layers) when a compositepresentation composed of these media clips or sequences is played back.One way of layering media content of media clips or sequences is tostack a media clip or a sequence of media clips above another media clipor sequence of media clips in the free-form space. For example, a mediaclip could be placed above a sequence of three media clips so that thelower edge of media clip would be aligned to the upper edge of thesequence. The bottom of a media clip can snap to the top of a sequencewhen the lower edge of the media clip is within a threshold distance ofthe top edge of the sequence.

For some embodiments, the free-form space also allows the user to editmedia clips in the free-form space. In some embodiments, the user canperform a trim operation on a selected a media clip to set a startand/or end point (i.e., in- and/or out-point) of the media clip. In somesuch embodiments, the trim operation can be performed by selecting andmoving a side edge of the media clip. For instance, the left edge of aselected media clip can be selected and moved (e.g., by performing adrag-and-drop operation) towards the right to set a start (i.e., anin-point) of the media clip. Similarly, the right edge of the selectedmedia clip can be selected and moved (e.g., by performing adrag-and-drop operation) towards the left to set an end point (i.e., anout-point) of the media clip.

In some embodiments, the trimming operation can be completed by using anappropriate side edge of the media clip to select (e.g., by releasingthe click in a drag-and-drop operation) a position along the length ofthe media clip to set as a starting or ending point. Some embodimentsgenerate a new media clip that represents the trimmed media clip.

Different embodiments allow the user to set the edit points of a mediaclip differently. For instance, instead of selecting and moving eitherside edge of a media clip to a new location, the user can invoke editcommands when the playhead is at a desired position along the length ofthe media clip during playback of the media clip. Specifically, when theplayhead reaches a desired position, the user can invoke an editingcommand to trim from the beginning of the media clip to the positionrepresented by the location of the playhead (e.g., a “cut to the playhead” command). In other embodiments, the user can drag the playhead toa desired position without playing back the media clip and then invokean editing command.

Some embodiments allow the user to split a media clip into two separatemedia clips in the free-form space. The user places a playhead over amedia clip when the media clip is being played back or when the usermanually selects a position along the length of the media clip. In someembodiments, the user can invoke a split command that divides theselected media clip into two separate media clips. One of the splitmedia clips represents a portion of the media content represented by theoriginal media clip from the beginning of the media content to theposition represented by the position of the playhead when the commandwas invoked, and the other split media clip represents the rest of themedia content represented by the original media clip (i.e., the portionof the media content starting from the position represented by theposition of the playhead to the end of the media content).

In addition to allowing the user to invoke editing commands, thefree-form space in some embodiments also allows the user to mark aregion of interest on a media clip. In some embodiments, the user canmark a region of interest on a media clip by invoking a command (e.g.,holding a keyboard key, selecting a UI item, etc.) to mark the start ofa region of interest and invoking another command to mark the end of theregion of interest while the playhead moves along the media clip. Insome embodiments, the marked region of interest is represented by amarker on the media clip that horizontally spans the start and end ofthe marked region of interest.

Different embodiments allow the user to mark a region of interestdifferently. For instance, instead of invoking two commands at thebeginning and the ending of the region of interest, the user can holddown a hotkey while the playhead moves from the beginning and the endingof the region. Alternatively, the user can click and hold down a UI iteminstead of the hotkey. In some embodiments, a media clip can be createdbased on a marked region of interest. The user selects a marker anddrags (e.g., using a drag-and-drop operation) the marker away from themedia clip to another location in the free-form space.

Moreover, some embodiments allow the user to tag at various positions ofa media clip to further organize the media clips in the free-form space.For example, a user can tag a frame of a video clip associated with amedia clip that shows a car with the tag “car,” “automobile,” or“transportation”. As such, the user can tag the content associated withmedia clip in any which way to further assist the user in organizing themedia clips in the free-form space.

Some embodiments allow the user to create a media clip based on a markedregion of interest. In some embodiments, the marker is a selectable UIitem for creating the media clip based on the marked region of interest.However, the region of the selectable UI item for creating the mediaclip based on the marked region of interest may be defined differentlyin different embodiments. For instance, the entire vertical section ofthe media clip that includes the marker may be defined as the selectableUI item for creating the media clip based on the marked region ofinterest. The user selects the selectable UI item that represents themarked region of interest and drags (e.g., using a drag-and-dropoperation) the marker away from the media clip to another location inthe display area to create a new media clip that includes the markedportion of the marked media clip. Instead of selecting and dragging aselectable UI item that represents the marked region of interest, someembodiments create a media clip from a marked region of interest througha copy and paste operation (e.g., by using a series of hotkeys,keystrokes, combinations of keystrokes, or options selected from apull-down or pop-up menu). Other methods are possible as well. Asmentioned above, some embodiments allow the user to tag positions of amedia clip. When a media clip that is created based on a marked regionof interest and the marked region of interest includes tags, someembodiments include the tags that are in the marked region of interestin the media clip that is created based on the marked region of interest(tags outside the marked region of interest are not included).

In addition, some embodiments provide additional tools for editing mediaclips, such as dynamic editing and playback of media clips in the space.In some embodiments, dynamic editing allows a user to perform operationson a media clip while the media clip is being played back. Examples ofdynamic editing operations include tagging instances in time of themedia clip, splitting the media clip into multiple media clips, trimmingthe ends of the media clip, and extending a trimmed media clip, amongother operations.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all inventive subject matter disclosed in this document.The Detailed Description that follows and the Drawings that are referredto in the Detailed Description will further describe the embodimentsdescribed in the Summary as well as other embodiments. Accordingly, tounderstand all the embodiments described by this document, a full reviewof the Summary, Detailed Description and the Drawings is needed.Moreover, the claimed subject matters are not to be limited by theillustrative details in the Summary, Detailed Description and theDrawing, but rather are to be defined by the appended claims, becausethe claimed subject matters can be embodied in other specific formswithout departing from the spirit of the subject matters.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purpose of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 conceptually illustrates a GUI of a media-editing application atseveral stages that show organizing graphical representations of mediaclips.

FIG. 2 illustrates an example GUI at several stages that show playingback a media clip in some embodiments.

FIG. 3 illustrates an example GUI at several stages that show playingback a media clip in some embodiments.

FIG. 4 illustrates an example GUI at several stages that showrearranging media clips in a sequence of media clips in someembodiments.

FIG. 5 illustrates an example GUI at several stages that show editingmedia clips in a sequence of media clips in some embodiments.

FIG. 6 illustrates an example GUI at several stages that show editingmedia clips in a sequence of media clips in some embodiments.

FIG. 7 illustrates an example GUI at several stages that show moving asequence of media clips in some embodiments.

FIG. 8 illustrates an example of data structure for a display area ofsome embodiments.

FIG. 9 illustrates an example of a free-form display area in someembodiments.

FIG. 10 illustrates an example GUI of a media-editing application ofsome embodiments.

FIG. 11 illustrates an example GUI of a media-editing application atseveral stages that show adding a sequence of media clips to one area ofthe GUI from the other area of the GUI.

FIG. 12 illustrates an example GUI of a media-editing application atseveral stages that show adding a sequence of media clips to one area ofthe GUI from the other area of the GUI.

FIG. 13 illustrates an example GUI of a media-editing application ofsome embodiments.

FIG. 14 illustrates an example GUI that shows moving a media clip insome embodiments.

FIG. 15 illustrates an example GUI that shows moving several media clipsin some embodiments.

FIG. 16 illustrates an example GUI at several stages that show pilingmedia clips in some embodiments.

FIG. 17 illustrates an example GUI at several stages that showcollapsing a media clip in some embodiments.

FIG. 18 illustrates an example GUI at several stages that showcollapsing media clips in some embodiments.

FIG. 19 illustrates an example GUI that shows skimming a media clip insome embodiments.

FIG. 20 illustrates an example GUI at several stages that show setting arepresentative frame of a media clip in some embodiments.

FIG. 21 illustrates an example of a conceptual process that someembodiments use to set a representative frame of a media clip.

FIG. 22 illustrates an example GUI at several stages that show playingback a media clip in some embodiments.

FIG. 23 illustrates an example GUI that show playing back a media clipin some embodiments.

FIG. 24 illustrates an example GUI that shows playing back and skimmingseveral media clips in some embodiments.

FIG. 25 illustrates an example GUI at several stages that show expandinga media clip into a filmstrip in some embodiments.

FIG. 26 illustrates an example media clip at several stages that showexpanding the media clip into a filmstrip in some embodiments.

FIG. 27 illustrates an example GUI at several stages that show playingback a media clip.

FIG. 28 illustrates an example GUI at several stages that showcompositing media clips in some embodiments.

FIG. 29 illustrates an example of data structure for a sequence of mediaclips in some embodiments.

FIG. 30 illustrates an example of a conceptual process that someembodiments use to snap one media clip to another.

FIG. 31 illustrates an example of snapping a media clip into a sequenceof media clips in some embodiments.

FIG. 32 illustrates an example GUI at several stages that show insertinga media clip into a sequence of media clips in some embodiments.

FIG. 33 illustrates an example GUI at several stages that show removinga media clip from a sequence of media clips in some embodiments.

FIG. 34 illustrates an example GUI at several stages that show removinga media clip from a sequence of media clips in some embodiments.

FIG. 35 illustrates an example GUI at several stages that show moving asequence of media clips in some embodiments.

FIG. 36 illustrates an example GUI at several stages that show adding asequence of media clips to another sequence of media clips in someembodiments.

FIG. 37 illustrates an example GUI that shows piling sequences of mediaclips in some embodiments.

FIG. 38 illustrates an example GUI at several stages that show stackingmedia clips in some embodiments.

FIG. 39 illustrates an example GUI at several stages that show snappinga sequence of media clips to another sequence of media clips in someembodiments.

FIG. 40 illustrates an example GUI at several stages that showcollapsing a sequence of media clips in some embodiments.

FIG. 41 illustrates an example GUI that shows skimming a sequence ofmedia clips in some embodiments.

FIG. 42 illustrates an example GUI at several stages that show skimminga sequence of media clips in some embodiments.

FIG. 43 illustrates an example of GUI that shows playing back a sequenceof media clips in some embodiments.

FIG. 44 illustrates an example of GUI at several stages that showplaying back of a sequence of media clips in some embodiments.

FIG. 45 illustrates an example of GUI at several stages that showplaying back of a sequence of media clips in some embodiments.

FIG. 46 illustrates an example of GUI at several stages that showrevealing names of media clips in some embodiments.

FIG. 47 illustrates an example of a conceptual process that someembodiments use to mark a region of interest in a media clip in someembodiments.

FIG. 48 illustrates an example media clip at several stages that showmarking a region of interest in a media clip in some embodiments.

FIG. 49 illustrates an example media clip at several stages that showmarking a region of interest in a media clip in some embodiments.

FIG. 50 illustrates an example of a conceptual process that someembodiments use to create a new media clip out of a marked media clip insome embodiments.

FIG. 51 illustrates an example GUI at several stages that show creatinga new media clip out of a marked media clip in some embodiments.

FIG. 52 illustrates an example GUI at several stages and in severalscenarios that show adjusting a marked region of a media clip in someembodiments.

FIG. 53 illustrates an example GUI at several stages that showduplicating a media clip in some embodiments.

FIG. 54 illustrates an example GUI at several stages that show tagging amedia clip and creating a bin in some embodiments.

FIG. 55 illustrates an example of a conceptual process that someembodiments use to tag a media clip and create a bin in someembodiments.

FIG. 56 illustrates an example GUI at several stages that show tagging amedia clip in some embodiments.

FIG. 57 illustrates an example GUI at several stages that show filteringmedia clips in some embodiments.

FIG. 58 illustrates an example of a conceptual process that someembodiments use to filter media clips.

FIG. 59 illustrates an example GUI at several stages that show filteringmedia clips in some embodiments.

FIG. 60 illustrates an example of data structure for a media clip insome embodiments.

FIG. 61 illustrates an example of a media clip at several stages thatshow setting a new in-point for the media clip in some embodiments.

FIG. 62 illustrates an example of a media clip at several stages thatshow setting a new out-point for the media clip in some embodiments.

FIG. 63 illustrates an example of a media clip at several stages thatshow setting a new in-point for the media clip in some embodiments.

FIG. 64 illustrates an example of a media clip at several stages thatshow setting a new out-point for the media clip in some embodiments.

FIG. 65 illustrates an example of a media clip at several stages thatshow extending an out-point for the media clip in some embodiments.

FIG. 66 illustrates an example of a media clip at several stages thatshow splitting the media clip in some embodiments.

FIG. 67 illustrates an example of a conceptual process that someembodiments use to edit media clips.

FIG. 68 illustrates an example of a sequence of media clips at severalstages that show setting a new in-point for a media clip in the sequencein some embodiments.

FIG. 69 illustrates an example of a sequence of media clips at severalstages that show trimming the sequence in some embodiments.

FIG. 70 illustrates an example of a sequence of media clips at severalstages that show setting a new in-point for a media clip in the sequencein some embodiments.

FIG. 71 illustrates an example of a sequence of media clips at severalstages that show extending an out-point for a media clip in the sequencein some embodiments.

FIG. 72 illustrates an example of a sequence of media clips at severalstages that show splitting a media clip in the sequence in someembodiments.

FIG. 73 illustrates an example of a sequence of media clips at severalstages that show marking a region of interest over several media clipsin the sequence in some embodiments.

FIG. 74 illustrates an example GUI at several stages that show setting anew in-point for a media clip in a sequence of media clips in someembodiments.

FIG. 75 illustrates an example GUI at several stages that show extendingan out-point of a media clip in a sequence of media clips in someembodiments.

FIG. 76 conceptually illustrates the software architecture of themedia-editing application of some embodiments.

FIG. 77 conceptually illustrates a process used by some embodiments todefine a media-editing application.

FIG. 78 illustrates an alternative GUI of the media-editing applicationof some embodiments.

FIG. 79 illustrates a computer system with which some embodiments of theinvention are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are set forth anddescribed. However, it will be clear and apparent to one skilled in theart that the invention is not limited to the embodiments set forth andthat the invention may be practiced without some of the specific detailsand examples discussed.

For a media-editing application that creates composite presentations,some embodiments of the invention provide a novel free-form space (e.g.,display area) that allows a user to both visually organize media clipsand composite media clips in order to create composite presentations. Insome embodiments, a media clip is a graphical representation of a pieceof media content that is stored on the computing device on which themedia-editing application executes or on a computing device to which themedia-editing application has access. Examples of such media contentinclude audio data, video data, text data, image data, and/or othermedia data. Accordingly, a media clip of some embodiments is any kind ofcontent clip (e.g., audio clip, video clip, text clip, picture clip, orother media clip) that can be used to create a composite presentation.

To visually organize media clips in the free-form space, someembodiments allow the user to add, remove, visually arrange andrearrange, and minimize the media clips. To composite media clips in thefree-form space, some embodiments also allow the user to createsequences of media clips, reorder the media clips within sequences,stack sequences, and add or remove media clips from sequences. In someembodiments, the media clips or created sequences of such media clipscan be placed at any location within the entire space when the user isorganizing compositing media clips. Compositing media clips, as usedhere and in the following sections of this application,

Some embodiments provide a free-form space that allows the user to placemedia clips (and sequences of media clips) in regions within the spacethat overlap each other at the same or different instances in time. Insome embodiments, a free-form space does not limit the placement ofmedia clips within the free-form space. That is, the media clips can beplaced at any location within the free-form space of such embodiments,as mentioned above.

In some embodiments, the free-form space is a trackless space thatallows the user to place media clips and sequences of media clipswithout being constrained by such tracks (e.g., the media clips andsequence of media clips do not have to be aligned along tracks). Inother words, the free-from space in some embodiments does not includetracks, such as those included in a typical timeline-based compositingarea of a media-editing application that is for compositing mediapresentations. In typical media editing applications, adjacent tracksare non-overlapping regions in which media clips can be placed withoutthe media clips overlapping each other. As mentioned above, thefree-form space of some embodiments media clips and sequences to beplaced in regions that overlap at the same or different instances intime.

Rather than use tracks, the free-form space of some embodiments is agridded space that includes gridlines on which the media clips align.Some embodiments define the gridlines of the space as fine as the pixelsof a display device displaying the space. In such embodiments, thedisplay area is referred to as gridless display area since a media clipplaced in any position in the display area aligns with the pixelgridlines. Other embodiments, however, define the gridlines of the spacemore coarse.

For the space of some embodiments, the positional relation between twoitems in the space (e.g., between two clips, between a sequence and aclip, between two sequences) does not necessarily define anyrelationship between the two items. For instance, because a timelinedoes not span along this space in some embodiments, the positionalrelationship between two items in the space does not necessarily definea temporal relationship between them. As another example, in someembodiments, the positional relationship between two items does notnecessarily define a logical relationship between the two items.However, for some items in the space, the positional relationship doesdefine a relationship when two items have been somehow associated witheach other (e.g., through user action). For example, as described above,a user can create a sequence of media clips (e.g., by concatenatingtogether several media clips). In such a concatenated sequence, thesemedia clips do have a temporal positional relationship with each other.Moreover, when two sequences are concatenated or composited together inthis space, the sequences (and the media clips in the sequences) have atemporal positional relationship with each other.

The embodiments described above mention numerous features of a free-formspace. However, other embodiments do not have to include all thefeatures mentioned above. For instance, although the free-form space isa trackless space in some embodiments, this space is divided intoseveral tracks in other embodiments in order to allow the user to bothvisually organize and composite media clips. Furthermore, in someembodiments, the free-form space is one of many spaces through which auser of the media-editing application organizes media clips. Otherspaces can be used to organize media clips as well. In otherembodiments, the free-form space is one of many spaces through which auser of the media-editing application composites media clips. Otherspaces, such as a timeline space, can be used to composite media clips.However, in some embodiments, the free-form space may not include allthe features and functionalities provided by the other spaces while inother embodiments the free-form space does include all the features andfunctionalities provided by the other spaces.

Several more detailed embodiments and examples will be described below.In many of these embodiments and examples, the space is a free-formspace that is for visually organizing media clips and compositing mediaclips. However, one of ordinary skill will realize that theseembodiments and examples can be implemented without some or all of thesefeatures. For example, in some embodiments, the free-form space is not atrackless space, but rather has several tracks that are used fororganizing and compositing the media clips.

For some embodiments of invention, FIG. 1 illustrates a graphical userinterface (“GUI”) 100 of a media-editing application that includes anorganizing and editing free-form space. Specifically, this figureillustrates the GUI 100 at six different stages 110, 120, 130, 140, 150,and 160 that show how the organizing and editing space can be used toorganize media clips. Each of these stages will be described in moredetail below after an introduction of the elements of GUI 100.

As shown in FIG. 1, the GUI 100 includes a media library 165, a displayarea 170, media clips (e.g., media clip 124) in the media library 165,and media clips (e.g., media clip 122) in the display area 170. Themedia library 165 is an area in the GUI 100 through which theapplication's user can select media clips to add to the display area 170in order to organize and edit the media clips. The display area 170 isan area in the GUI 100 where a user can add, remove, organize, visuallyarrange, collapse, and play back media clips, for example. As shown,media clips may be displayed with different appearances or dimensions indifferent areas of the GUI 100. For instance, the media clips in thedisplay area 170 in this example are represented using thumbnails (i.e.,a set of equally-sized images) while the media clips in the medialibrary 165 are represented using smaller thumbnails (i.e., a set ofequally-sized, but smaller, images). In other embodiments, the mediaclips in differently areas of the GUI 100 are defined any number ofdifferent ways.

The operation of the GUI 100 will now be described by reference to thestate of this GUI during the six stages 110, 120, 130, 140, 150, and 160that are illustrated in FIG. 1. In the first stage 110, the display area170 displays media clips 122, 126, 128, and 132. These clips have beenpreviously added to the display area 170 by a user of the application.The media clips 126, 128, and 132 are placed near each other in thedisplay 170 area to illustrate an example of media clips visuallygrouped together.

Additional media clips can be added to the display area 170 from themedia library 165 at any time. These clips can be added to the displayarea 170 in several different ways. For instance, in some embodiments,the user can add a media clip from the media library 165 by firstselecting a media clip in the media library 165. The selected media clipis then added to the display area 170 through a drag-and-drop operation(e.g., by using a cursor-control device, such as a mouse or trackpad ora finger for a touch-screen device, to select this media clip from themedia library 165 and drag it to the display area 170). As anotherexample, a media clip from the media library 165 can be added to thedisplay area 170 by selecting the media clip in the media library 165and then selecting an option from a pull-down menu or a pop-up menu (notshown). The first stage 110 illustrates a media clip being added to thedisplay area 170. In this stage, a user selects (e.g., clicking on,tapping) the media clip 124 in the media library 165. This selection isindicated by the bold border of the media clip 124. The user then addsthe media clip 124 to the display 170 by performing a drag-and-dropoperation as indicated by the arrow 145.

In the second stage 120, the display area 170 displays the media clip124 that was added in the first stage 110 in addition to the media clips122, 126, 128, and 132. As shown, the media clip 124 is placed near themedia clip 122, and the two media clips form a visual grouping in thedisplay area 170. In this stage, there are two visual groupings—onegrouping that includes the media clips 126, 128, and 132 and anothergrouping that includes the media clips 122 and 124. In the display area170, these groupings can be rearranged to form different visualgroupings. For instance, the user can move the media clip 132 near themedia clips 122 or 124 to form a new visual grouping that includes themedia clips 122, 124, and 132. Thus, the user can visually group themedia clips into any number of different visual groupings that the userdesires.

In some embodiments, media clips in a visual grouping can be rearrangedso that the user can change the visual arrangement of the media clips inthe visual grouping. The third stage 130 illustrates an example of suchrearranging of media clips in a visual grouping. In this stage, the usermoves the media clip 126 (e.g., through a drag-and-drop operation)between the media clips 128 and 132 as indicated by the arrow 180. Insome embodiments, the media clips 128 and 132 automatically slidehorizontally away from each other, as shown by arrows, to make room forthe media clip 126 when the media clip 126 is placed between them. Inother embodiments, the media clips 128 and 132 do not move when themedia clip 126 is placed between the media clips 128 and 132. In suchembodiments, the media clip 126 will overlap the media clips 128 and132.

The fourth stage 140 illustrates the GUI 100 after the completion of therearrangement of the media clips 126, 128, and 132 in the third stage130. As shown, the media clips 126, 128, and 132 are now rearranged inthe order of 128, 126, and 132 from left to right. In addition to visualgroupings, the media clips in the display area 170 can be grouped intological groupings. For instance, in some embodiments, the user canselect several media clips in the display area 170 and create a logicalgrouping that includes the selected media clips by executing a keycommand (e.g., control-G), a combination of key commands, a hotkey, orany other appropriate method. Some embodiments require that media clipsto be within a particular distance of each other before they can belogically grouped while other embodiments allow media clips locatedanywhere in the free-form space to be logically grouped together. Once alogical grouping of media clip is created, the logical grouping can bemoved within the space as a single item. For instance, performing adrag-and-drop operation on one of the logically grouped media clips alsomoves the other media clips in the logical group in the same relativedirection and distance as the medial clip on which drag-and-dropoperation is performed.

In some embodiments, media clips can be minimized or collapsed into asmaller thumbnail or an icon in the display area. A media clip can beminimized or collapsed in several different ways. For instance, a usercan minimize or collapse a media clip by selecting the media clip andthen selecting a minimize button that is displayed on or close to themedia clip. In some embodiments, the user selects a menu option that isdisplayed in a pull-down menu that is accessed by selecting a top-levelmenu in the GUI or displayed in a pop-up menu that is accessed through acursor control operation (e.g., a right-click cursor operation or acontrol-click cursor operation).

In the fifth stage 150, the GUI 100 shows an example of a media clipbeing minimized in the display area 170. The stage shows the media clip124 after a right-click cursor operation has been performed on the mediaclip 124 to cause a pop-up menu 185 that includes a minimize command tobe displayed. The pop-up menu 185 includes a play option and a minimizeoption in addition to other options (not shown) Although the pop-up menu185 in this example shows a set of particular options, one of ordinaryskill will recognize that other embodiments include different menuoptions and/or commands in the pop-up menu 185. This stage also showsthe user selecting the minimize option to shrink the media clip 124 intoa smaller thumbnail. The sixth stage 160 illustrates the media clip 124after it has been minimized. As shown, the minimized media clip 124 isnow represented by a smaller sized rectangle and is moved to the lowerleft corner of the GUI 100. The size and location of the media clip 124before it was minimized is depicted as a dotted rectangle. In someembodiments, the application provides an animation to illustrate thesize minimization and movement of the media clip 124.

Some embodiments allow a user to play back media clips in a free-formspace of a media-editing application. For instance, FIGS. 2 and 3illustrate two different methods of playing back media clips in afree-form display area of some embodiments. Specifically, FIG. 2 showsplaying back a media clip through a set of playback controls, and FIG. 3shows playing back a media clip through a menu item selection. FIG. 2will now be described followed by a description of FIG. 3.

FIG. 2 illustrates a GUI 200 at four different stages 225, 230, 235, and240 of a media clip playback operation. The GUI 200 is similar to theGUI 100 described above by reference to FIG. 1, except the GUI 200includes a set of playback controls 245-255 for playing back a mediaclip in the display area 270 and a viewing area 275 for displayingplayback of a media clip. As shown, the set of playback controls 245-255includes a user selectable UI item 245 (e.g., backward playback button245) for backward playback of a media clip, a user selectable UI item250 (e.g., pause button 250) for pausing playback of the media clip, anda user selectable UI item 255 (e.g., forward playback button 255) forforward playback of the media clip.

In some embodiments, playback of the selected media clip is displayedwithin the selected media clip while in other embodiments playback ofthe selected media clip is displayed in a viewing area (e.g., viewingarea 275). In yet other embodiments, playback of the selected media clipis displayed both within the selected media clip itself and a viewingarea, which is illustrated by FIGS. 2 and 3.

Some embodiments display a playhead (or scrubber bar) over a selectedmedia clip that moves horizontally along the selected media clip as itis playing back. The position of the playhead along the displayed lengthof the media clip indicates the position of the content being played.For example, if the media clip is a video clip that includes a sequenceof frames, the position of the playhead along the length of the mediaclip represents the position of the currently displayed frame in thesequence of frames of the video clip.

In some embodiments, the rate at which the playhead moves horizontallyalong the selected media clip being played is different for media clipsthat have different durations. This is because, in some embodiments, thedisplayed length of a media clip represents the duration of the contentassociated with the media clip with the left edge representing thestarting point and the right edge representing the ending point of thecontent and the media clips displayed in the free-form space areuniform-sized rectangles. For example, in such embodiments, a playheadmoves horizontally along a three-minute audio clip faster than aplayhead along a five-minute video clip since the duration of the audioclip is shorter than the duration of the video clip.

As mentioned above, the viewing area 275 is an area in the GUI 200 whereplayback of a media clip is displayed. As shown, the viewing area 275 islocated outside the display area 270. In some embodiments, however, theviewing area 275 is located within the display area 270 (not shown). Theviewing area 275 is shared with other workspaces included in themedia-editing application in some embodiments. For example, media clipsin other workspaces are also played back in the viewing area 275.Furthermore, the viewing area 275 of some embodiments is expanded (e.g.,by using a keystroke, a combination of keystrokes, a hotkey, a pull-downor pop-up menu selection, etc.) to occupy the entire monitor or displaydevice (i.e., full screen mode playback) of the computing device onwhich the media-editing application is being executed.

The operation of the GUI 200 will now be described by reference to thestate of this GUI during four different stages 225, 230, 235, and 240that are illustrated in FIG. 2. In the first stage 225, the display area270 displays media clips 122, 124, 126, 128, and 132. These clips havebeen previously added to the display area 270 by a user of theapplication. None of the media clips are being played back in thisstage.

In the second stage 230, the user selects (e.g., by clicking on,tapping) the media clip 124 to play back, as indicated by the bold edgesof media clip 124. In the third stage 235, the user selects (e.g., byclicking on, tapping) the forward play button 255 to start playback ofthe media clip 124. The fourth stage 240 shows the GUI 200 after thestart of the playback of the media clip 124. As mentioned above, thisexample displays the playback of the media clip 124 within the mediaclip 124 and in the viewing area 275. As shown, the current position ofthe playback of the media clip 124 in this stage is indicated by theposition of the playhead 260 along the displayed length of the mediaclip 124.

FIG. 3 illustrates another method of playing back a media clip in afree-form display area 370 of a GUI 300. This figure shows the GUI 300,which is similar to the GUI 200 illustrated in FIG. 2, of amedia-editing application of some embodiments. FIG. 3 shows the GUI 300in two different stages 335 and 340 of a media clip playback operation.In the first stage 335, the user selects (e.g., by clicking on, tapping)a play option displayed in a pop-up menu 185. In some embodiments, thepop-up menu 185 is accessed through a cursor control operation (e.g., aright-click cursor operation or a control-click cursor operation) thatis performed on the media clip 124, which the user selects as the mediaclip to play back. The second stage 340 illustrates the GUI 300 duringthe playback of the media clip 124 after the play option was selectedfrom the pop-up menu 185. In this stage, playback of the media clip 124is displayed in the viewing area 275 as well as within the media clip124. As noted above, a playhead 350 indicates the position of thecontent being played. In some embodiments, playback of the media clip124 is only displayed in the viewing area 275 (i.e., playback is notdisplayed within the selected media

As mentioned above, the free-form display area of some embodimentsallows the user to create sequences of media clips, reorder the mediaclips within sequences, stack sequences, and add or remove media clipsfrom sequences in addition to allowing a user to add, remove, organize,visually arrange, collapse, and play back the media clips. In someembodiments, a sequence of media clips is a group of media clipsassociated in a particular chronological order. Some embodiments form asequence of media clips by horizontally concatenating media clips in thedisplay area. For example, a sequence that includes a first media clipand a second media clip is formed when the second media clip is placedor moved in the display area such that the left edge of the second mediaclip coincides with the right edge of the first media clip. When playingback the sequence, the first media clip is first played back and thenthe second media clip is played back.

FIG. 4 illustrates examples of operations performed on media clips andsequences in a free-form display area 170 of GUI 100. Specifically, thisfigure illustrates the GUI 100 in five different stages 410, 420, 430,440, 450, and 460 of the operations. The GUI 100 is similar to the GUI100 described above by reference to FIG. 1. In FIG. 4, the first stageillustrates an operation for creating a sequence of media clips. Thisstage shows a display area 170 that displays media clips 122, 124, 444,126, 128, and 132. As shown, the media clips 126, 128, and 132 form asequence of media clips. In this stage, the user selects (e.g., byclicking on, tapping) the media clip 124 in the media library 165 andadds it to the display area 170 (e.g., by performing a drag-and-dropoperation). The bold border of media clip 124 indicates the selection ofthe media clip 124. As the user adds the media clip 124 to the displayarea 170, the user places it near the right side of media clip 122 toform a sequence.

In some embodiments, when a media clip is moved within a thresholddistance of another media clip in the display area, the media clip beingmoved “snaps” to the other media clip in order to form a sequence.Referring to the first stage 410 as an example, as the media clip 124 isadded to the display area 170 and as the left edge of the media clip 124moves close to the right edge of the media clip 122, the left edge ofthe media clip 124 and the right edge of the media clip 122 “snaps”together. Further details of this snapping feature will be describedbelow. The second stage 420 illustrates the GUI 100 after the completionof the creation of the sequence formed by the media clips 122 and 124.At this stage, the display area 170 displays two sequenceschronologically ordered from left to right: a sequence formed by themedia clips 126, 128, and 132, and a sequence formed by the media clips122 and 124. Some embodiments chronologically order the media clipsincluded in a sequence from right to left. However, for purposes ofexplanation and clarity, the sequences described in this application arechronologically ordered from left to right.

As mentioned above, a sequence is a group of media clips associated in aparticular chronological order. Some embodiments allow the user toreorder the media clips in a sequence in order to change thechronological order of the media clips in the sequence. The third stage430 illustrates an example operation for reordering media clips in asequence of media clips. This stage shows the user moving the media clip132 from the right end of the sequence to between the media clips 126and 128 (e.g., by performing a drag-and-drop operation) as indicated byan arrow. In some embodiments, when a media clip is placed between twomedia clips in a sequence, the two media clips automatically slidehorizontally away from each other in order to make a room for the mediaclip being placed between the two media clips. The third stage 430illustrates such a sliding feature with the two arrow pointing away fromeach other under the media clips 126 and 128. The fourth stage 440illustrates the GUI 100 after the reordering operation is completed. Thechronological order of the sequence from left to right is now from themedia clip 126 to 132 to 128.

The fifth stage 450 shows an operation for adding a media clip to asequence of media clips. In this stage, a media clip 444, which is notpart of any sequence, is added into a sequence formed by media clips126, 132, and 128. To add the media clip 444 to this sequence, the userselects the media clip 444 and moves it between the media clips 126 and132 (e.g., by performing a drag-and-drop operation) in the sequence.Like the media clips 126 and 128 in the third stage 430, the media clips126 and 132 automatically slide horizontally away from each other inorder to make a room for the media clip 444 to be added. The sixth stage460 illustrates the GUI 100 after the completion of the additionoperation. This stage shows the sequence of the media clips 126, 444,132, and 128 chronologically ordered from left to right.

Some embodiments allow the user to layer media content of a media clipor a sequence of media clips over media content of another media clip orsequence of media clips in a composite presentation composed of thesemedia clips or sequences of media clips. In some such embodiments, mediacontent in the one layer (e.g., the top layer) is played back over anymedia content in the other layers (e.g., the lower layers) when acomposite presentation composed of these media clips or sequences isplayed back. For instance, in some embodiments, the media content in thetop layer is played back over any other content in the lower layerswhile in other embodiments, the media content in the bottom layer isplayed back over any other content in the upper layers. One example wayof layering media content of media clips or sequences is to stack amedia clip or a sequence of media clips above another media clip orsequence of media clips. For example, the media clip 444 could be placedabove the sequence of media clips 126, 128, 132 in the first stage 410so that the lower edge of media clip 444 would be aligned to the upperedge of the sequence. Like the snapping feature described above, thebottom of a media clip can snap to the top of a sequence when the loweredge of the media clip is within a threshold distance of the top edge ofthe sequence. In such example, if the media clip 444 is place above themedia clip 128 of the sequence, the media content (e.g. image frames)represented by the media clip 444 will be displayed over the mediacontent represented by the media clip 128 when a composite presentationthat includes the media clip 444 and the sequence that includes themedia clip 128 is played back in some embodiments.

For some embodiments, the free-form display area allows the user to editmedia clips in the display area. FIG. 5 illustrates examples of trimmingand splitting operations within this display area in some embodiments.Specifically, this figure illustrates a GUI 500 in five different stages510, 520, 530, 540, and 550 of the operations for editing media clips.The GUI 500 is similar to the GUI 100 described above by reference toFIG. 1, except the GUI 500 includes a user selectable UI item 580 (e.g.,split button 580) for activating a splitting tool to split a media clipinto several media clips.

The first stage 510 illustrates a sequence of media clips that includesmedia clips 126, 128, and 132. As shown by the bold border, the userselects (e.g., by clicking on, tapping) the media clip 126. In someembodiments, the user can perform a trim operation on a selected a mediaclip to set a start and/or end point (i.e., in- and/or out-point) of themedia clip. In some such embodiments, the trim operation can beperformed by selecting and moving a side edge of the media clip. Forinstance, the left edge of a selected media clip can be selected andmoved (e.g., by performing a click-and-drag operation) towards the rightto set a start (i.e., an in-point) of the media clip. Similarly, theright edge of the selected media clip can be selected and moved (e.g.,by performing a click-and-drag operation) towards the left to set an endpoint (i.e., an out-point) of the media clip.

The second stage 520 illustrates the GUI 500 in the middle of a trimoperation. In this stage, the user has selects and moves the left edgeof the media clip 126 towards the right, as indicated by an arrow. Insome embodiments, a border with the original dimensions of the mediaclip is displayed in addition to the border that is being moved, asshown in this stage.

The third stage 530 illustrates the GUI 500 after the completion of thetrimming operation. In some embodiments, the trimming operation can becompleted by using an appropriate side edge of the media clip to select(e.g., by releasing the click in a click-and-drag operation) a positionalong the length of the media clip to set as a starting or ending point.Some embodiments generate a new media clip that represents the trimmedmedia clip. As shown, a new media clip 534 is generated and replaces themedia clip 126 in the sequence. In addition, the displayed length of themedia clip 534 remains the same as the displayed length of the mediaclip 126 even though the durations of the media clips 534 and 126 aredifferent. This is because the media clips illustrated in FIG. 5 arerepresented by uniform-sized rectangles, as described above and below insome embodiments.

Different embodiments allow the user to set the edit points of a mediaclip differently. For instance, instead of selecting and moving eitherside edge of a media clip to a new location, the user can invoke editcommands when the playhead is at a desired position along the length ofthe media clip during playback of the media clip. Specifically, when theplayhead reaches a desired position, the user can invoke an editingcommand to trim from the beginning of the media clip to the positionrepresented by the location of the playhead (e.g., a “cut to the playhead” command). In other embodiments, the user can drag the playhead toa desired position without playing back the media clip and then invokean editing command.

The fourth stage 540 shows the GUI 500 before an operation for splittinga media clip into two separate media clips. As mentioned above, aplayhead is displayed over a media clip when the media clip is beingplayed back or when the user manually selects a position along thedisplayed length of the media clip. As shown, a playback head 555 isdisplayed over the media clip 128. In this example, the media clip 128is playing back. In some embodiments, the user can invoke a splitcommand (e.g., by selecting the split button 580) that divides theselected media clip into two separate media clips. One of the splitmedia clips represents a portion of the media content represented by theoriginal media clip from the beginning of the media content to theposition represented by the position of the playhead 555, and the othersplit media clip represents the rest of the media content represented bythe original media clip (i.e., the portion of the media content startingfrom the position represented by the position of the playhead 555 to theend of the media content).

The fifth stage 550 illustrates the GUI 500 after the split operation iscompleted. As illustrated in this stage, two media clips 536 and 538area generated to represent the two media clips that resulted from thesplit operation performed on the media clip 128 in the fourth stage 540.In some embodiments, the media clips 126 and 132 slide horizontally awayfrom each other in order to make a room for the media clips 536 and 538in the sequence, as indicated by the arrow below the media clip 132pointing to the right. Similar to the media clip 534 illustrated thethird stage 530, the displayed length of the media clips 536 and 538remains the same as the displayed length of the media clip 128 despitethe durations of the media clips 536 and 538 being different than theduration of the media clip 128. Again, this is because the media clipsillustrated in FIG. 5 are represented by uniform-sized rectangles.

FIG. 6 illustrates that the free-form display area in some embodimentsalso allows the user to mark a region of interest on a media clip andcreate a new media clip based on the marked region on the media clip, inaddition to allowing the user to invoke editing commands. Specifically,this figures illustrates a GUI 600 at five different stages 610, 620,630, 640, and 650 that show the creation of a region of interest of amedia clip 132 and the use of this region of interest to create anothermedia clip 642. The GUI 600 is similar to the GUI 100 described above byreference to FIG. 1, except the GUI 600 includes a user selectable UIitem (e.g., region button 660) for marking a region of interest on amedia clip.

The first stage 610 illustrates the free-form display area 670 as themedia clip 132 is being played back within its thumbnail representation.During this playback, a playhead 655 is displayed over the media clip132 to indicate the position of the playback of the media clip 132. Insome embodiments, the user can mark a region of interest on a media clipby invoking a command (e.g., holding a keyboard key, selecting a UI itemsuch the region button 660, etc.) to mark the start of a region ofinterest and invoking another command to mark the end of the region ofinterest while the playhead moves along the media clip. In this example,the user selects (e.g., by clicking on, tapping) the region button 660to start marking a region of interest and selects (e.g., by clicking on,tapping) the region button 660 again to stop marking the region ofinterest while the playhead 655 moves along the displayed length ofmedia clip 132. Accordingly, the user selects the region button 660 inthis stage.

Different embodiments allow the user to mark a region of interestdifferently. For instance, instead of invoking two commands (i.e., oneat the beginning and another at the ending of the desired region ofinterest), the user can hold down a hotkey while the playhead movesalong the displayed length of the media clips playing back in order tomark a region of interest in some embodiments. Alternatively, the usercan click and hold down the region button 660 instead of the hotkey.

In some embodiments, the marked region of interest is represented by amarker on the media clip that horizontally spans the start and end ofthe marked region of interest. The second stage 620 illustrates a marker675 that represents the marked region of interest. At this stage, theuser has just selected the region button 660 a second time to indicatethe end of the region of interest. The third stage 630 illustrates theGUI 600 after the completion of the marking the region of interest. Inthis stage, the marker 675 represents the entire marked region ofinterest.

In some embodiments, a media clip can be created based on a markedregion of interest. The fourth stage 650 illustrates such an operation.In this stage, the marker 675 is a selectable UI item for creating themedia clip based on the marked region of interest. However, the regionof the selectable UI item for creating the media clip based on themarked region of interest may be defined differently in differentembodiments. For instance, the entire vertical section of the media clip132 that includes the marker may be defined as the selectable UI itemfor creating the media clip based on the marked region of interest. Theuser selects the marker 675 and drags (e.g., using a drag-and-dropoperation) the marker 675 away from the media clip 132 to anotherlocation in the display area 670 as indicated by a dotted arrow. Insteadof selecting and dragging a selectable UI item that represents themarked region of interest, some embodiments create a media clip from amarked region of interest through a copy and paste operation (e.g., byusing a series of hotkeys, keystrokes, combinations of keystrokes, oroptions selected from a pull-down or pop-up menu).

The fifth stage 650 shows the GUI 600 after the completion of theoperation in the fourth stage 640. Media clip 642 represents the mediaclip created as a result of the operation. That is, the media clip 642represents the portion of the content of media clip 132 that was markedwith the marker 675.

Some embodiments allow the user to tag at various positions of a mediaclip to further organize the media clips in the display area. Forexample, a user can tag a frame of a video clip associated with a mediaclip that shows a car with the tag “car,” “automobile,” or“transportation”. As such, the user can tag the content associated withmedia clip in any which way to further assist the user in organizing themedia clips in the display area, which is described in further detailbelow. In some embodiments, tags included in a marked region of interestare also included in a media clip that is created based on the markedregion of interest (tags outside the marked region of interest are notincluded).

In addition, some embodiments provide additional tools for editing mediaclips, such as dynamic editing and playback of media clips in thefree-form space. In some embodiments, dynamic editing allows a user toperform operations on a media clip while the media clip is playing back.Examples of dynamic editing operations include tagging instances in timeof the media clip, splitting the media clip into multiple media clips,trimming the media clip, and extending a trimmed media clip, among otheroperations.

Several more detailed embodiments of the invention are described in thesections below. Section I provides a conceptual description of theattributes of the display areas of some embodiments that allow the userto organize and edit media clips in the display areas of someembodiments. Next, Section II describes an organizational aspect of thedisplay areas of some embodiments. Section III describes compositingmedia clips within the display areas of some embodiments. In addition,Section III describes several operations that a user can perform withcomposited media clips. Next, Section IV follows this with a descriptionof editing operations that can be performed on a single media clip orcomposited media clips. Section V describes the software architecture ofan application that employs the display areas of some embodiments. Next,Section VI describes the process used to define the media-editingapplication of some embodiments. Finally, Section VII describes acomputer system that implements some embodiments of the invention.

I. Display Area

As mentioned above, display areas of the media-editing application ofsome embodiments have several attributes that allow a user of theapplication to organize and edit media clips and sequences of mediaclips within these areas. Several more detailed examples of suchattributes will now be described below.

As mentioned above, some embodiments provide a free-form space thatallows the user to place media clips (and sequences of media clips) inregions within the space that overlap each other at the same ordifferent instances in time. In some embodiments, a free-form space doesnot limit the placement of media clips within the free-form space. Thatis, the media clips can be placed at any location within the free-formspace of such embodiments, as mentioned above.

In some embodiments, the free-form space is a trackless space thatallows the user to place media clips and sequences of media clipswithout being constrained by such tracks (e.g., the media clips andsequence of media clips do not have to be aligned along tracks). Inother words, the free-from space in some embodiments does not includetracks, such as those included in a typical timeline-based compositingarea of a media-editing application that is for compositing mediapresentations. In some media editing applications, a track is a regiondefined by two parallel horizontal lines in a compositing space.Typically, there are multiple tracks in such media-editing application.Media clips or sequences can be placed in a track and are aligned toeither of the two parallel horizontal lines defining the track. Mediaclips and sequences cannot vertically overlap within a track. Wherethere are multiple tracks, neither a media clip nor a sequence canoccupy multiple tracks at once.

In some embodiments, a display area of some embodiments allows the userto move media clips and sequences in the display area such that regionsthat the media clips and sequences occupy after being moved can overlapwith the regions the media clips and sequences used to occupy beforebeing moved. FIG. 7 illustrates an example of a region-overlap aspect ofa free-form display area 700 of some embodiments. Specifically, thisfigure illustrates sequences positioned at different locations in thedisplay area 700. As shown, FIG. 7 illustrates the display area 700 atfive different stages 710, 720, 730, 740, and 750. The display area 700is an area of a GUI (not shown) for an application in which a userplaces media clips and sequences.

The first stage 710 shows sequences 755 and 760 positioned in thefree-form display area 700. The sequence 755 includes two media clipsand the sequence 760 includes three media clips. At this stage 710, noneof the sequences 755 and 760 are being moved within the display area 700

The second stage 720 illustrates the movement of the sequence 760 withinthe trackless free-form display area 700. Specifically, the sequence 760is moved (e.g., by performing a drag-and-drop operation) slightly up andto the right from its previous position, which is indicated by a dottedsequence of rectangles. As shown, the region in which the sequence 760is positioned partially overlaps with the region in which it used bepositioned within the display area 700.

The third stage 730 illustrates another movement of the sequence 760within the free-form display area 700. Similarly, the sequence 760 ismoved (e.g., by performing a drag-and-drop operation) slightly up and tothe right from its location within the free-form display area 700 in thesecond stage 720. In this stage, the region in which the sequence 760 ispositioned still partially overlaps with the region in which it wasoriginally positioned in the first stage 710 (and the second stage 720).Thus, these stages illustrate how the sequence 760 can be positioned inmultiple regions within the free-form display area 700 that overlap withthe region in which the sequence 760 was originally positioned in thefirst stage 710.

The fourth stage 740 shows the movement of the sequence 755 within thefree-form display area 700. The sequence 755 is moved (e.g., byperforming a drag-and-drop operation) down and to the left from itslocation within the free-form display area 700 in the stages 710-730. Inparticular, the sequence 755 is moved to a region that partiallyoverlaps the region in which the sequence 760 was originally positionedin the first stage 710.

The tracks of typical media-editing applications have a temporalcomponent. In particular, the tracks are defined to span a single commontimeline. For example, a particular media clip (or sequence) on a trackis chronologically ordered before any media clips or sequencespositioned to the right of the particular media clip (even those ondifferent tracks) on the timeline. Therefore, when the compositepresentation is played back, the particular media clip is played backbefore any media clips or sequences positioned to the right of theparticular media clip on the timeline.

However, a free-form display area of some embodiments does not have sucha temporal component. Referring to FIG. 7 as an example, the first stage710 shows the sequence 760 positioned to the left of the sequence 755.Unlike a space with tracks and a timeline of typical media-editingapplications, the sequence 755 can be played back before the sequence760 and vice versa. In fact, the sequences 755 and 760 in the displayarea 700 are disjointed and temporally unrelated: they are two separateand independent sequences.

A. Display Area Data Structure

FIG. 8 illustrates a data structure 800 for a free-form display area ofsome embodiments. The media-editing application creates the datastructure 800 when the media-editing application is opened for executionin some embodiments. In other embodiments, a data structure is createdwhen a first media clip is placed into an empty free-form display areaof the application. The data structure 800 includes different pieces ofinformation that define the media clips in the display area. As shown,the data structure 800 includes a list of media clips that are displayedin the display area and coordinates for each media clip that determinethe location of the media clip in the display area. In some embodiments,the coordinates are expressed in a Cartesian coordinate system format(e.g., distance x, distance y) while in other embodiments thecoordinates are expressed in a polar coordinate system format (e.g.,radius r, angle θ). Other two-dimensional coordinate system formats arepossible in other embodiments.

Each media clip in the list is a reference to a data structure for theparticular media clip in some embodiments. As shown, the reference forclip 1 links to a data structure 805 for the media clip. The datastructure 805 includes a clip ID field that uniquely defines the mediaclip, a reference to a source media file (e.g., an audio file, a videofile), an in-point for the source file, and an out-point for the sourcefile. In some embodiments, the default in-points and out-points are thebeginning and the end of the whole duration of the media clip. In somesuch embodiments, the in-point and out-point store an in-point and anout-point within the timecode of the source media file that respectivelyserves as the beginning and the end of the media content represented bythe media clip. The reference to the media source file for media clip 1links to a data structure 810, as shown in FIG. 8. The data structure810 includes media data. Examples of media data include audio data,video data, text data, image data, and/or other media data.

When a media clip is added to the display area, information about thenewly added media clip is added to the data structure 800. Conversely,when a media clip is removed from the display area, information aboutthe media clip is removed from the data structure. When a media clip inthe display area is moved, the coordinates corresponding to the movedmedia clip are updated with the coordinates of the new position of mediaclip in the display area.

Different embodiments may include different information in a datastructure for a free-form display area. For instance, some embodimentsmight include a data field that defines the resolution of the displayarea. Some embodiments might include a data field that defines thegranularity of a grid for the display area. Although the data structure800 shows a list of references to media clips, the list can includereferences to sequences, such as the sequence data structure describedin further detail below by reference to FIG. 29, and coordinates thatdetermine the location of corresponding sequences in the display area.

B. Gridded Display Area

In some embodiments, the free-form display area is a gridded displayarea. The gridded display area of some embodiments is a display areathat positions media clips within the display area by using a grid(e.g., a set of evenly spaced and interleaving horizontal and verticalgridlines) that is coarser than the pixel grid used to display the userinterface. In other words, each tile (e.g., the area encompassed by apair of consecutive horizontal gridlines and a pair of verticalgridlines) of such a grid includes more than one pixel. In a griddeddisplay area, media clips and sequences of media clips are positionedsuch that one or more edges (e.g., all edges) of the media clips andsequences of media clips are aligned with the gridlines of the grid. Inaddition, media clips and sequences of media clips can span multipleconsecutive gridlines (i.e., occupy multiple tiles of the grid) andstill be aligned with the gridlines of the grid. In such cases, twomedia clips can be positioned so that the regions they occupy(horizontally, vertically, or both) overlap each other at same ordifferent instance in time.

In other embodiments, the free-form display area is a gridless displayarea. In these embodiments, the grid of the gridless display area is notcoarser than the grid formed by the pixels used to display the userinterface. That is, gridlines of the grid are defined by the pixelsthemselves. In such a gridless display area, any position of media clipsand sequences of media clips align with the gridlines defined by thepixels. Like the gridded display area of some embodiments, a griddeddisplay area allows two media clips to be positioned so that the regionsthey occupy overlap each other at same or different instance in time.

FIG. 9 illustrates an example of such a gridded display area 900 of someembodiments. This figure illustrates the display area 900 at fourdifferent stages 910, 920, 930, and 940. The display area 900 is similarto the display area 700 described above by reference to FIG. 7, exceptthe display area 900 includes gridlines. The display area 900 displayssequences of media clips 955 and 960, and a media clip 965. The displayarea 900 also displays dashed lines show the gridlines of the displayarea 900. However, the gridlines do not have to be shown as dashed lines(e.g., solid lines) in some embodiments. In fact, some embodiments donot display the gridlines of the gridded display area.

In the first stage 910, the sequence 955 is positioned in a region ofthe display area 900 defined by three rows of the grid betweenhorizontal gridlines 3 and 6 as shown. The sequence 960 is positioned ina region of the display area 900 also defined by three rows of the gridbut from horizontal gridline 4 to horizontal gridline 7. The media clip965 is positioned in a region defined by three rows of the grid fromhorizontal gridline 8 to horizontal gridline 11. As shown, the positionsof the media clips and sequences displayed in the display area 900 aredefined by the gridlines of the display area 900 (i.e., they are alignedto the vertical and horizontal gridlines).

The second stage 920 illustrates the display area 900 after the sequence955 is moved up (e.g., by performing a drag-and-drop operation) by onerow of the grid and is now positioned in a region defined by three rowsof the grid between horizontal gridlines 4 and 7. As shown, the positionof the sequence 955 in this stage overlaps with its position in thefirst stage 910.

The third stage 930 shows the display area 900 after the sequence 955 ismoved up (e.g., by performing a drag-and-drop operation) again by onerow of the grid and is now positioned in a region defined by three rowsof the grid between horizontal gridlines 5 and 8. In this stage, theposition of the sequence 955 overlaps its position in the first andsecond stages 910 and 920.

In the fourth stage 940, the sequence 960 is moved instead of thesequence 955. In this stage, the sequence 960 is moved (e.g., byperforming a drag-and-drop operation) and positioned in a region thatoverlaps the region defined by the position of the sequence 955 in thefirst stage. As shown, the sequence 960 is positioned in a regiondefined by three rows of the grid between horizontal gridlines 1 and 4.As illustrated by the exemplary positioning of sequences in FIG. 9,media clips and sequences displayed in a gridded display area can bepositioned at any position in the display area defined by the gridlinesof the grid including positions that overlap each other and positionsthat overlap their previous positions.

C. Secondary Organizing Space

As noted above, some embodiments provide a free-form display area as oneof several workspaces through which the user organizes media clips,composite media clips, or both. FIG. 10 illustrates a GUI 1000 of amedia-editing application of some embodiments that includes a free-formdisplay area 1030 as one of several organizing spaces. As shown in FIG.10, the GUI 1000 includes a viewing area 1010, a browser 1020, and thedisplay area 1030. The viewing area 1010 is an area in the GUI 1000 fordisplaying playback of a media clip. For instance, a user can select amedia clip in browser 1020 or in the display area 1030 and play back theselected media clip in the viewing area 1010. In some embodiments, thebrowser 1020 is the primary area for organizing media clips andsequences of media clips, and the display area 1030 is the secondaryorganizing area. In other embodiments, the display area 1030 is insteadthe primary area for organizing media clips and sequences of mediaclips, and the browser 1020 is the secondary organizing area.

The browser 1020 is an area in the GUI 1000 for organizing media clips.The user can create bins (or folders) in the browser to organize mediaclips in any way the user desires. Media clips can be added to thebrowser 1020 by selecting (e.g., by clicking on, tapping) a media clipfrom a media library, such as the one illustrated in FIG. 1, and placing(e.g., by performing a drag-and-drop operation) the media clip in one ofthe bins. In addition, the user can select and move (e.g., by performinga drag-and-drop operation) media clips in the browser 1020 from one binto another bin in the browser 1020. The display area 1030 is an area inthe GUI 1000 for visually organizing media clips as previouslydescribed. Thus, media clips can be added to the display area (e.g., byselecting them from a media browser), visually grouped together, andcollapsed (i.e., minimized). Furthermore, media clips can be added fromone organizing space to another. For instance, a media clip in thebrowser 1020 can be selected and moved (e.g., by performing adrag-and-drop operation) into the display area 1030. Similarly, a mediaclip in the display area 1030 can be selected and moved (e.g., byperforming a drag-and-drop operation) into the browser 1020 (e.g., byperforming a drag-and-drop operation on the media clip). As such, theviewing area 610 and the display area 1030 provide the user with twodifferent organizing spaces of the GUI 1000 for organizing media clipsdifferently.

D. Secondary Compositing Space

FIG. 11 illustrates a GUI 1100 of a media-editing application of someembodiments that includes a display area 1115 as one of severalcompositing spaces in the GUI 1100. As shown in FIG. 11, the GUI 1100includes the display area 1115, a composite display area 1120, and aviewing area 1130. The viewing area 1130 is for displaying playback ofmedia clips and sequences in the display area 1115 and the compositedisplay area 1120. In some embodiments, the composite display area 1120is the primary area for compositing media clips, and the display area1115 is the secondary compositing area. In other embodiments, thedisplay area 1115 is the primary area for compositing media clips ratherthan the composite display area 1120, and the composite display area1120 is the secondary compositing area.

The composite display area 1120 includes multiple tracks that span asingle timeline and displays a graphical representation of the compositepresentation (also called a “composite representation”) by displayingmedia clips (or sequences) that form the composite presentation. Assuch, the media clips in the composite display area 1120 have positionalrelationships among each other. Specifically, the media clips placed inthese multiple tracks have temporal relationships relative to each otherbased on the single timeline. As mentioned above, a particular mediaclip is chronologically ordered before any media clips (or sequences)positioned to the right of the particular media clip in the compositepresentation based on the timeline.

In the GUI 1100, the display area 1115 is an area like the display area700 described above by reference to FIG. 7. The display area 1115 allowsa user of the media-editing application to composite media clips byforming sequences with media clips. Since there is no timeline or formaltracks in the display area 1115, the sequences formed in the displayarea 1115 do not have any positional relationships with each other. Inother words, the sequences displayed in the display area 1115 aredisjointed and temporally unrelated. In some embodiments, the user canfreely experiment with different compositing ideas in the display area1115 and then bring the composited media clips into the compositedisplay area 1120 to fine-tune the resulting composite presentation.

FIG. 11 illustrates the GUI 1100 at two different stages 1105 and 1110of such an operation. The first stage 1105 shows the user selecting andmoving (e.g., by performing a drag-and-drop operation) a sequence ofmedia clips 1125 in the display area 1115 into the composite displayarea 1120, as indicated by a dotted arrow. The second stage 1110 showsthe GUI 1100 after the sequence 1125 is moved into the composite displayarea 1120. In particular, the sequence 1125 is moved to video track 5 inthe composite display area 1120. As shown, the three media clipsincluded in the sequence 1125 are displayed in the composite displayarea 1120 in the same chronological order. Since the composite displayarea 1120 includes a timeline, the media clips in the composite displayarea 1120 are shown with different lengths, which represent thedurations of the different media clips. As illustrated in this stage,moving the sequence 1125 from the display area 1115 to the compositedisplay area 1120 does not remove the sequence 1125 from the displayarea 1115. In this manner, the user can fine-tune the compositepresentation and continue to experiment with the sequence 1125 in thedisplay area 1115. In some embodiments, selecting and moving thesequence 1125 from the display area 1115 to the composite display area1120 removes the sequence from the display area 1115.

FIG. 12 conversely illustrates the user moving a sequence from acomposite display area to a display area in order to make edits to thesequence in the display area. As shown, this figure illustrates a GUI1100 at two different stages 1205 and 1210 of such operation. Thecomposite display area 1120 displays an additional sequence 1225 in thecomposite presentation.

In the first stage 1205, the user selects and moves (e.g., by performinga drag-and-drop operation) a sequence 1225 from the composite displayarea 1120 to the display area 1115. The second stage 1210 illustratesthe GUI 1100 after the sequence 1225 is moved to the display area 1115.In this stage, the three media clips included in the sequence 1225 aredisplayed in the display area 1115 in the same chronological order asshown in the first stage 1205.

In addition, the media clips and sequences of media clips arearepresented in the display area 1115 differently than in the compositedisplay area 1120 in some embodiments, as shown in this figure. Forinstance, the media clips in the display area 1115, which are shownhaving different durations in the composite display area 1120, are shownwith the same displayed length since media clips are represented byuniform-sized rectangles in the free-form display area of someembodiments (even if the media clips have different durations), asdescribed above. On the other hand, the displayed lengths of the mediaclips in the composite display area 1120 are represented based on theduration of the media clips relative to the timeline. For example, thedisplayed length of a two-minute media clip will span two minutes alongthe timeline in the composite display area 1120.

This stage also shows that moving the sequence 1225 from the compositedisplay area 1120 into the display area 1115 does not remove thesequence 1225 from the composite display area 1120. This way, the usercan use both the display area 1115 to experiment with the sequence 1225and the composite display area 1120 to fine-tune the compositepresentation. Some embodiments, however, remove the sequence 1225 fromthe composite display area 1120 after the sequence 1225 is moved to thedisplay area 1115.

E. Secondary Organizing and Compositing Space

FIG. 13 illustrates a GUI 1300 of a media-editing application of someembodiments that includes a display area 1305 as one of severalorganizing and compositing spaces in the GUI 1300. As shown, the GUI1300 includes the display area 1305, a browser 1310 that is similar tothe one illustrated in FIG. 11, a composite display area 1315 similar tothe one illustrated in FIG. 12, and a viewing area 1320. The viewingarea 1320 is for displaying playback of media clips and sequences in thedisplay area 1305, the browser 1310, and the composite display area1315.

The browser 1310 is similar to the browser 1020 described above byreference to FIG. 10, except the browser 1310 includes additionalfeatures. For instance, media clips in the browser can be selected andmoved (e.g., by performing a drag-and-drop operation) into the compositedisplay 1315 to be part of a composite presentation. The compositedisplay 1315 is similar to the composite display area 1120 describedabove by reference to FIG. 11, except the composite display 1315includes additional features. For example, media clips in the compositedisplay 1315 can be selected and moved (e.g., by performing adrag-and-drop operation) into the browser 1310.

The display area 1305 includes the features and functionalities of thedisplay area 1030 described above by reference to FIG. 10 and thefeatures and functionalities of the display area 1115 described above byreference to FIGS. 11. Thus, the GUI 1300 allows the user to use thebrowser 1310 and display area 1305 to organize media clips and thedisplay area 1305 and the composite display area 1315 to composite mediaclips.

Similar to many of the display areas described above, the display area1305 is, in some embodiments, a separate display area (e.g., a window)in a GUI of a media-editing application that allows items in the displayarea to be placed at any location within the entire display area whenthe user visually organizes and composites items (e.g., media clips andsequences) in the display area. In some embodiments, the display area1305 is a two dimensional, bounded, and contiguous space in which itemscan be placed at any location. Some embodiments define the display area1305 to not include any partitions that divide the display area intosmaller partitions. In some embodiments, items cannot be placed between(straddle) multiple partitions in a display area that includespartitions. In some embodiments, a display area does not encompass twoor more other display areas that have boundaries within the displayarea.

Moreover, one of ordinary skill will recognize that the display area1305 of some embodiments could include typical display area functions.For instance, the display area 1305 of some embodiments includes userselectable UI items for closing, minimizing, and maximizing the displayarea (not shown). In addition, display areas can be resized, hidden,restored, and moved. While many features of a display area are describedabove, other embodiments of the display area include additionalfeatures, different features, or a combination of features. Referring toFIG. 13, the display area 1305, the browser 1310, the composite displayarea 1315, and the viewing area 1320 are examples of different andseparate display areas of the GUI 1300.

Like the free-form display areas described above, such as 170, 700, 900,etc., items (e.g., media clips and sequences) in a free-form displayarea of some embodiments do not necessarily have any definedrelationship among each other. FIG. 13 illustrates examples of suchunrelated items. Specifically, the figure shows multiple items, which donot have any defined relationship among each other, concurrentlydisplayed in a free-form display area (i.e., display area 1305). Forinstance, sequence 1325 does not have any defined relationship with theother media clips in the display area 1305, such as media clip 1330,1335, 1340, and 1345 among other media clips in the display area 1305.Accordingly, the display area 1305 of some embodiments allows multipleitems that do not have any defined relationship among them to beconcurrently displayed in the display area 1305. Furthermore, thedisplay area 1305 of some such embodiments also allows unrelated itemsdisplayed in the display area 1305 to be concurrently placed atlocations in the display area 1305 such that the unrelated items share acommon area in the display area 1305. One such example is illustrated bythe placement of the media clips 1335 and 1340. As shown, the upperright corner of the media clip 1335 and the lower left corner of themedia clip 1340 share a common area (i.e., they overlap) in the displayarea 1305.

Having described a display area that has several attributes that enablesa user of a media-editing application to visually organize and compositemedia clips, the following Section II will now describe in detail someoperations that the user can perform using the display area in order tovisually organize media clips.

II. Organizing and Viewing in Display Area

Several operations that a user of a media-editing application canperform to organize and play back media clips are described below.Examples of these operations include moving, piling, collapsing,skimming, and playing back media clips in a display area of someembodiments of the invention.

A. Moving Media Clips

FIG. 14 illustrates an example of moving media clips in a free-formdisplay area 700 of some embodiments. Specifically, FIG. 14 illustratesthe moving of a media clip to organize it in the display area 700. Asshown, the display area 700 displays media clips 1415-1460. Before themedia clip 1460 was moved, it was visually grouped by itself, asillustrated by a dotted rectangle that indicates the previous positionof the media clip 1460. The media clip 1460 is selected and moved (e.g.,by performing a drag-and-drop operation) next to the media clip 1455, asindicated by the dotted arrow, to visually group it with the media clips1450 and 1455.

The display area 700 of some embodiments allows the user to visuallygroup media clips together. As shown, the media clips 1415-1460 arevisually grouped into several groups. Specifically, the media clips1415-1425, 1430-1435, 1440-1445, and 1450-1460 are visually groupedtogether. As shown, media clips can be visually grouped in any number ofdifferent ways. For instance, the media clips 1415-1425 are positionedtogether vertically, but they are not vertically aligned. Similarly, themedia clip 1440 and 1445 are positioned together horizontally, but theyare not horizontally aligned. Further, the media clips 1430-1435 and1450-1460 are positioned together in a horizontally-aligned manner.Although the media clips 1415-1460 are positioned separately (i.e., theydo not overlap each other), the media clips 1415-1460 can be positionedto overlap each other. By allowing the user to visually group mediaclips in the display area 700, the user can organize media clips in thedisplay area 700 in a visually meaningful way to the user.

Some embodiments allow the user to visually organize sequences of mediaclips by moving multiple media clips at once. FIG. 15 illustratesanother example of moving media clips in a free-form display area 1500of some embodiments. This figure illustrates the movement of multiplemedia clips in the display area 1500 in two different stages 1505 and1510. As shown, the display area 1500 displays media clips 1535-1550 andother media clips. The display area 1500 is similar to the free-formdisplay area 700 described above by reference to FIG. 7.

In the first stage 1505, the user selects the media clips 1535-1545,which is indicated by the dotted rectangle surrounding the media clips,and moves the selected media clips down and to the left within thedisplay area 1500, which is indicated by the dotted hollow arrow.Different embodiments provide different methods for the user to selectmultiple media clips. In some embodiments, the media clips areindividually selected (e.g., clicking on, tapping) while holding down akeyboard key or a hotkey. In some embodiments, the media clips areselected using a cursor-controlled device (e.g., a mouse, a trackball, atrackpad) to draw a selection box around the media clips (not shown).The user then performs a drag-and-drop operation on the selected mediaclips in order to place the selected media clips in a desired locationwithin the display area.

The second stage 1510 illustrates the user moving the selected mediaclips to a new location within the display area 1500 next to the mediaclip 1550. The user in this example moves the selected media clips nextto the media clip 1550 to create a visual grouping of media clips1535-1550. The dotted rectangles indicate the previous locations of themedia clips 1535-1545.

B. Piling Media Clips

Another way to organize media clips in a free-form display area of someembodiments is to pile the media clips together. FIG. 16 illustrates anexample of the piling of media clips together in a free-form displayarea 700 of some embodiments. Specifically, this figure illustrates thepiling of media clips together in the display area 700 at four differentstages 1610-1640. As shown, the display area 700 displays a pile ofmedia clips 1660 that includes four media clips, media clips 1645-1655,and other media clips.

The display area 700 allows the user to pile media clips together inorder to create visual groupings and/or formal groupings. For example,the pile of media clips 1660 is a pile of media clips already created bythe user. By creating the pile of media clips 1660, some embodimentsautomatically create a corresponding logical structure (e.g., a bin or afolder) that includes the media clips included in the pile of mediaclips 1660 while other embodiments do not create a corresponding logicalstructure (i.e., the pile of media clips 1660 is only a visualgrouping).

The first stage 1610 shows the display area 700 before a piling of mediaclips operation. At the second stage 1620, the user selects the mediaclip 1645 and piles it on top of the media clip 1650 (e.g., byperforming a drag-and-drop operation) similar to the selection andmovement of the media clip 1460. As shown, the media clip 1645 is nowpiled on top of the media clip 1650. At this stage a pile of media clips1645 and 1650 is created.

In the third stage 1630, the user selects the media clip 1655 and pilesit on top of the media clips 1645 and 1650 (e.g., by performing adrag-and-drop operation) also similar to the selection and movement ofthe media clip 1460. This stage shows the media clip 1655 added to thepile of media clips 1645 and 1650 in the second stage 1620. The fourthstage 1640 shows the display area 700 after the piling operationsillustrated in the second and third stages 1620 and 1630.

As shown in FIG. 16, the user can create a pile of media clips in orderto create a visual grouping and/or formal grouping of the three mediaclips. Moreover, the user can add media clips to the pile of mediaclips, add media clips to other piles of media clips (e.g., the pile ofmedia clips 1660), remove media clips from piles of media clips, andeven pile sequences, as described in further detail below by referenceto FIG. 37. Accordingly, piling media clips together provides the userwith another way to visually group and/or formally group media clips ina free-form display area.

C. Collapsing Media Clips

In some embodiments, the user is allow to collapse (or minimize) mediaclips in a free-form display area 700 of some embodiments. FIG. 17illustrates an example of such collapsing operation of a media clip inthe free-form display area 700 of some embodiments. This figureillustrates the display area 700 at three different stages 1710-1730 ofthe collapsing operation. As shown, the display area 700 displays mediaclip 1735 as well as other media clips.

The display area 1700 allows the user to minimize or collapse a mediaclip into a smaller display of the media clip or an icon in order tovisually organize the media clips and/or reduce clutter in the displayarea 1700. In some embodiments, the collapsed media clips areautomatically placed along the periphery of the display area 1700. Somesuch embodiments automatically place a collapsed media clip along theedge of the display area 1700 to which the media clip is closest whenthe media clip is collapsed. In other embodiments, a collapsed mediaclip is placed next to the position of the media clip before the mediaclip is actually collapsed. Alternatively, some embodiments allow theuser to collapse a media clip by dragging the media clip and dropping itnear one of the edges of the display area 1700. In these embodiments,when any part of media clip is placed within a threshold distance froman edge of the display area 1700 or in contact with the edge, the mediaclip collapses. Some such embodiments would require to disable anyscrollbars that automatically scrolls the display area when a media clipor a cursor is placed near an edge of the display area.

Moreover, the user can select collapsed media clips to restore the mediaclips to their full size in some embodiments. In some such embodiments,moving a collapsed media clip away from the periphery of the displayarea causes the media clip to be restored to their full size.

The first stage 1710 illustrates the media clip 1735 that includes aselectable minimize item 1745. In some embodiments, the selectableminimize item 1745 is displayed on a media clip when a cursor (or anyother type of position indicator) is within a threshold distance of themedia clip 1735. In other embodiments, the selectable minimize item 1745is displayed when the cursor moves over the media clip 1735. In yetother embodiments, the selectable minimize item 1745 is displayed onlywhen the user selects (e.g., clicking on, tapping) the media clip 1735.Furthermore, different embodiments display the item 1745 at differentlocations. For instance, the item 1745 is displayed in the lower leftcorner of the media clip 1735 in some embodiments.

At the second stage 1720, the user moves the cursor over the selectableminimize item 1745 and selects (e.g., clicking on, tapping) the item1745, as indicated by a highlighting of the item 1745. In someembodiments, the media clip 1735 is minimized by using a hotkey, akeystroke, a combination of keystrokes, an option selected from apull-down or pop-up menu, or any other appropriate method. The thirdstage 1730 shows the media clip 1735 minimized into icon 1740 as aresult of the selection of the selectable minimize item 1745 in thesecond stage 1720.

As described above, some embodiments allow the user to minimize mediaclips individually. Some embodiments also allow the user to minimizemultiple media clips at once.

FIG. 18 illustrates an example of a collapsing operation of a group ofmedia clips in a free-form display area 700 of some embodiments. Thisfigure illustrates the display area 700 at four different stages1810-1840 of the collapsing operation. As shown, the display area 700displays media clips 1845-1855 and other media clips.

The display area 700 allows the user to minimize or collapse a group ofmedia clip into smaller displays of the media clips or icons. In someembodiments, collapsed groups of media clips are placed in a similarfashion as described above by reference to FIG. 17. In addition, theuser can select collapsed groups of media clips to restore the mediaclips to their full size in some embodiments.

In the first stage 1810, the user selects the media clips 1845-1855similar to the selection of the media clips 1535-1545. The selection isindicated by the dotted rectangle surrounding the three media clips. Insome embodiments, the dotted rectangle remains displayed after the mediaclips are selected, as illustrated in this stage. The second stage 1820illustrates a selectable minimize item 1860 displayed along the upperright corner of the dotted rectangle. Different embodiments display theitem 1860 at different locations. For example, the item 1860 isdisplayed in the lower left corner of the dotted rectangle in someembodiments. The conditions that cause the minimize item 1860 to bedisplayed is similar to the conditions that cause the minimize item 1745to be displayed.

In the third stage 1830, the user moves the cursor over the selectableminimize item 1860 and selects it, as indicated by the highlighting ofthe item 1860, similar to the selection of the item 1745. The fourthstage 1740 shows the group of the media clips 1835-1845 minimized into agroup of icons 1865 and located on the right edge of the display area700. As shown, a dotted circle is displayed around the group of icons1865 and a selectable restore item 1870 is displayed near the top leftportion of the dotted circle. The user can select the item 1870 torestore the group of icons 1865 to their full size.

D. Skimming Media Clips

Some embodiments allow a user to skim (or scrub) through a selectedmedia clip in a free-form display area by dragging a playhead throughthe media clip. As the user drags the playhead across the media clip,the frame of the media clip that temporally corresponds to the locationof the playhead in the media clip is displayed within the media clip. Inaddition, the user can skim media clips to identify a frame to representthe media clip (i.e., a poster frame) as described in further detailbelow by reference to FIG. 20.

In some embodiments, instead of selecting and dragging the playheadthrough the media clip, the user can skim a media clip simply by movinga cursor over the media clip without selecting the media clip. In suchembodiments, the playhead is positioned at the location in the mediaclip where the cursor is placed.

FIG. 19 illustrates an example of a skimming operation that may beperformed on a media clip in a free-form display area 700 of someembodiments. As shown, the display area 700 displays a media clip 1910,other media clips, and a cursor 1930. The media clip 1910 includes aplayhead 1935. FIG. 19 also illustrates the media clip 1910 at fivedifferent stages 1940-1960 of the skimming operation.

As mentioned, the user can skim a media clip by moving a cursor over themedia clip without selecting the media clip. The example skimmingoperation will be described by reference to this method. At the firststage 1940, the user uses the cursor 1930 to move the playhead 1925 overthe media clip 1910 at approximately one-sixth of the way through themedia clip 1910. The frame of the media clip 1910 that represents thelocation of the playhead 1935 is displayed within the media clip 1910.The frame displays a person shortly after hitting a golf ball towards ahole on a green.

In the second stage 1945, the user uses the cursor 1930 to move theplayhead 1935 further to the right at approximately halfway through themedia clip 1910 and the frame of the media clip 1910 that represents thelocation of the playhead 1935 is displayed within the media clip 1910.The frame displayed in this stage shows the golf ball on the green. Thethird stage 1950 shows the media clip 1910 after the user uses thecursor 1930 to move the playhead 1935 to the frame that is a little pasthalfway through the media clip 1910. As shown, this frame shows the golfball about to go in the hole on the green.

At the fourth stage 1955, the user uses the cursor 1930 to move theplayhead 1935 at approximately five-sixths through the media clip 1910,and the frame of the media clip 1910 that represents the location of theplayhead 1935 is displayed in the media clip 1910. This frame shows theperson and green shortly after the ball rolled into the hole on thegreen. At the last stage 1960, the user uses the cursors 1930 to movethe playhead 1935 near the last frame of the media clip 1910. This frameshows the person celebrating after the ball rolled in the hole on thegreen.

While skimming a media clip, some embodiments provide a method for theuser to set a frame in the media clip as a representative frame of themedia clip (also referred to as a poster frame or a representativethumbnail) in order to select a frame that better represents thecontents of the media clip. In some embodiments, a representative frameof a media clip is a video frame (e.g., frame or field) of the mediaclip that is the visual representation of the media clip in the displayarea when the media clip is not played or skimmed. Some embodiments usethe first frame of a media clip as the poster frame of the media clipwhile other embodiments use the middle frame of the media clip. Thus,different embodiments define the default poster frame of media clipsdifferently.

FIG. 20 illustrates an example of setting a frame of a media clip as arepresentative frame of the media clip in a free-form display area 700of some embodiments. This figure illustrates the free-form display area700 at six different stages 2010-2060 of the frame setting operation. Asshown, the display area 700 displays a media clip 2065, other mediaclips, and a cursor 2070.

The first stage 2010 illustrates the display area 2000 before the startof the frame setting operation. At the second stage 2020, the user movesthe cursor 2070 over the media clip 2065 and moves a playhead 2075 to aframe near the beginning of the media clip 2065 in a similar manner tothe skimming operation described above by reference to FIG. 19. Theframe shows a skateboarder at the bottom of a half-pipe. In the thirdstage 2030, the user continues to skim the media clip 2065 and moves theplayhead 2075 to a frame at approximately two-fifths through the mediaclip 2065. This frame shows the skateboarder near the top of the leftside of the half-pipe.

The fourth stage 2040 illustrates the user using a hotkey to set theframe in the third stage 2040, which shows the skateboarder near the topof the left side of the half-pipe, as the poster frame. Although thisexample shows the user using hotkey to set the poster frame, otherembodiments use different methods, such as using a keystroke, acombination of keystrokes, an option selected from a pull-down or pop-upmenu, or any other appropriate method. At the fifth stage 2050, the useruses the cursor 2070 to move the playhead 2075 to at approximatelythree-quarters through the media clip 2065. This frame shows theskateboarder in midair in the middle of a fall. At the last stage 2060,the user is no longer skimming the media clip 2065. As shown, the posterframe of the media clip 2065 is the frame set in the fourth stage 2040.

FIG. 21 illustrates a process 2100 of some embodiments for setting aposter frame for a media clip. In some embodiments, the process 2100 isperformed when the user inputs a command, such as the one illustrated inFIG. 20, while skimming a media clip (or playback of the media clip).The process 2100 starts by receiving (at 2105) a “set poster-frame”command. Different embodiments define different commands to setposter-frame of a media clip. For instance, some embodiments use ahotkey, a keystroke, a combination of keystrokes, an option selectedfrom a pull-down or pop-up menu, or any other appropriate command.

Next, the process identifies (at 2110) the selected frame position inthe media clip. In some embodiments, the position of the selected framein the media clip is expressed as a timecode in the source fileassociated with the media clip. The process 2100 then sets (at 2115) theidentified frame as the poster-frame of the media clip in the media clipdata structure, such as the media clip data structure illustrated abovein FIG. 8.

E. Playing Back Media Clips

Some embodiments allow a user to play back a media clip in a free-formdisplay area. A playhead similar to the one displayed during a skimmingoperation moves across the media clip as it is being played back. Insome embodiments, the playhead moves across the media clip at differentspeeds based on the duration of the media clip. For instance, if a firstmedia clip is five minutes long and a second media clip is ten minuteslong, the playhead on the first media clip moves across the media cliptwice as fast in the display during playback as during playback of thesecond media clip. In some embodiments, a playhead is not displayedduring playback of a media clip.

FIG. 22 illustrates an example of a playback operation of a media clipin a free-form display area 700 of some embodiments. Specifically, FIG.22 illustrates media clip 2230 at four different stages 2210-2240 of theplayback operation. As shown, the display area 700 displays a media clip2260, other media clips, and a cursor 2265. The first stage 2210 showsthe display area 700 before the start of the playback operation.

At the second stage 2220, the user selects (e.g., by clicking on,tapping) the media clip 2260 using the cursor 2265. In some embodiments,a set of playback controls 2245-2255 are displayed when the user selectsthe media clip 2260. The set of playback controls is similar to the setof playback controls 245-255. That is, the selectable UI item 2245 isfor backward playback, the selectable UI item 2250 is for pausingplayback, and the selectable UI item 2255 is for forward playback. Insome embodiments, selecting the UI item 2245 while a media clip isalready playing backwards increases the speed of the backward playback.Conversely, selecting the UI item 2255 while a media clip is alreadyplaying forwards increases the speed of the forward playback in someembodiments. Further, the direction of playback can be changed byselecting the opposite playback button while a media clip is beingplayed back in a particular direction.

At the third stage 2230, the user selects the UI item 2255 to play themedia clip 2260 forward. Different embodiments implement playbackcontrol commands differently. For instance, hotkeys can be assigned tothe playback control commands so that pressing a hotkey invokes thecorresponding playback control command. Alternatively, a set ofdrop-down menu items may replace or be used in conjunction with theplayback controls 2245-2255. In some embodiments, a playhead 2270 isdisplayed on the media clip 2260 when the media clip 2260 is played back(forward or backwards) to indicate the frame of the media clip that isbeing played back. In this stage, the playhead 2270 indicates a framenear the beginning of the media clip 2260 is being played. The fourthstage 2240 shows the display area 700 near the end of the playback ofthe media clip 2260, as indicated by the position of the playhead 2270on the media clip 2260. In some embodiments, the set of playbackcontrols 2245-2255 disappear after the playback of a media clip isfinished.

FIG. 23 illustrates another example of a playback operation of a mediaclip in a free-form display area 2305 of a GUI 2300. As shown, the GUI2300 includes the display area 2305 and a viewing area 2310. The viewingarea 2310 is for displaying playback of a media clip in the display area2305. The display area 2305 displays a media clip 2315 that includes aplayhead 2320 as well as displaying other media clips. The display area2300 is similar to the display area 700 described above by reference toFIG. 7. FIG. 23 also illustrates the media clip 2320 at six differentstages 2325-2350 of the playback operation.

The first stage 2325 shows the media clip 2315 at the start of theforward playback of the media clip 2315. As mentioned above, differentembodiments implement playback controls, such as hotkeys, keystrokes,combinations of keystrokes, selections from a pull-down or pop-up menu,or any other appropriate method. The second through sixth stages2330-2350 illustrate the progress of the playback of the media clip2315, which is indicated at each stage by the position of the playhead2320 on the media clip 2315. In some embodiments, the playback of themedia clip 2315 is only displayed within the media clip 2315 while inother embodiments the playback of the media clip 2315 is only displayedin the viewing area 2310. In yet other embodiments, the playback of themedia clip 2315 is displayed in both.

FIGS. 22 and 23 both illustrate the playback of one media clip. Someembodiments also allow the user to simultaneously playback and skimmultiple media clips. FIG. 24 illustrates an example of simultaneousplayback and skimming of multiple media clips in a free-form displayarea 2305 of a GUI 2300. The display area 2310 displays media clips2415-2425 and other media clips.

As shown in FIG. 24, the media clips 2420 and 2425 are playing back andthe media clip 2415 is being skimmed by the user. To start the playbackof the media clips 2420 and 2425, the user successively selects andinvokes a command to start playback of the media clips 2420 and 2425similar to the way playback is started for media clip 2315. After theplayback is started for the media clips 2420 and 2425, the user skimsthe media clip 2415 similar to the skimming of 1910 while the mediaclips 2420 and 2425 are still playing back.

In some embodiments, the viewing area 2310 displays the playback orskimming of the media clip that the user most recently started orskimmed. In this example, since the user is still skimming the mediaclip 2415, the frame being skimmed by the user is displayed in theviewing area 2310. If a media clip is not being skimmed, playback of themost recently started media clip is displayed in the viewing area 2310.

FIG. 25 illustrates an example of expanding a media clip into afilmstrip in a free-form display area 2305 of a GUI 2300. As shown, thedisplay area 2305 displays a media clip 2525 as well as other mediaclips.

The display area 2305 also allows the user to expand a media clip into afilmstrip. As mentioned above, the media clips in the display area arerepresented as uniformly-sized rectangles in some embodiments. Afilmstrip is another way to represent media content of a media clip. Insome embodiments, a filmstrip is displayed in the display area as aseries of horizontally concatenated rectangles each of which displays aframe of the media clip. The number of rectangles in a filmstripdisplayed in the display area is pre-determined in some embodiments. Insuch embodiments, the media content of the media clip is divided intothe pre-determined number of segments and each segment is represented bya rectangle in a filmstrip. As such, a filmstrip effectively is a seriesof chronologically ordered sub-clips of one media clip.

When the user is skimming or marking a region of interest of a mediaclip represented as a filmstrip, because the media content of the mediaclip is represented by a longer length of the filmstrip, the user isable to locate a frame more precisely than the user is able to when themedia clip is represented as a single-rectangle. When not being skimmedor played back, the filmstrip displays the poster-frame of each segmentin each rectangle in some embodiments. In addition, a filmstrip isskimmed and played back in the same manner as a media clip representedas a single rectangle in some embodiments. Also, a region of interest ofa media clip can be marked on a filmstrip in the same manner as a mediaclip represented as a single rectangle is marked. Once the user is doneviewing, the user may choose to restore the media clip back to itsoriginal single-rectangle representation.

FIG. 25 illustrates the GUI 2300 at two different stages 2505 and 2510of the filmstrip operation. At the first stage 2505 of the GUI 2300, theuser invokes a command to expand the media clip 2525 into a filmstrip.In some embodiments, the user invokes the command by performing any oneof the following operations: a double-click operation on the media clip2535, a keystroke, a combination of keystrokes, and an option selectionfrom a pull-down or pop-up menu. Other embodiments do not allow all ofthese operations and only allow a subset of these operations to performto invoke the command. Yet other embodiments provide other appropriateoperations to perform to invoke the command.

The second stage 2510 of the GUI 2300 shows the media clip 2525 expandedinto a filmstrip 2530. In this example, a filmstrip is defined to havesix segments. As such, the display area 2305 displays the six segmentsof the media clip 2525 as six rectangles that each display aposter-frame of the corresponding segments.

FIG. 26 illustrates the media clip 2525 and its filmstrip 2530 ingreater detail. The displayed length of the media clip 2525 representsthe duration of the media clip 2525, which is six seconds in thisexample. Six bi-directional arrows 2605-2630 that together span thedisplayed length of the media clip 2525 each represent one second of themedia clip. When represented as the filmstrip 2530, each of the sixone-second segments of the media clip 2525 is represented by a rectanglein the filmstrip 2530. As such, the duration of each rectangle in thefilmstrip 2530 is one second.

FIG. 27 illustrates an example of a playback operation of a media clipin a full screen mode in a free-form display area 2705 of a GUI 2700. Asshown, the GUI 2700 includes a viewing area 2710, a set of playbackcontrols 2720-2730, and the display area 2705. The viewing area 2710 issimilar to the viewing area 2300 and the set of playback controls2720-2730 is similar to the set of playback controls 245-255. Thedisplay area 2705 displays a media clip 2735 and other media clips aswell. The display area 2705 is similar to the display area 2305described above by reference to FIG. 23.

The display area 2705 also allows the user to playback a media clip inthe display area in a full screen mode. In the full screen mode, themedia clip being played back is displayed over the entire GUI 2700 insome embodiments. Other embodiments play back a media clip over theentire display area 2705 in the full screen mode. In yet otherembodiments, playback of a media clip in the full screen mode displaysthe media clip over the entire screen (not shown) or display area of thedevice on which a media-editing application that provides the displayarea 2705 executes.

In some embodiments, the user can switch the playback of a media clip toa full screen mode by invoking a command, such as using a hotkey, akeystroke, a combination of keystrokes, an option selection from apull-down or pop-up menu, or any other appropriate method. Someembodiments allow the media clip to be played back in full screen modebefore or after the media clip has already begun playback in the displayarea. Other embodiments only allow playback of a media clip in the fullscreen mode if the command is invoked before the playback of the mediaclip starts. Yet other embodiments only allow playback of a media clipin the full screen mode after the playback of the media clip starts. Insome such embodiments, the poster frame of the selected media clip isdisplayed in full screen mode before playback begins. In someembodiments, a playhead does not appear in the full screen mode.

FIG. 27 illustrates the GUI 2700 at two different stages 2740 and 2745of the full screen mode playback operation. At the first stage 2740, theuser selects the media clip 2735 and then selects the selectable fullscreen item 2715. The user then selects the selectable forward play item2730 to start playback of the selected media clip 2735. The second stage2745 shows the GUI 2700 after playback of the media clip 2735 in fullscreen mode starts. As shown, the playback of the media clip 2735 isdisplayed over the entire GUI 2700.

Having described several operations that a user of a media-editingapplication can perform to organize and play back media clips in adisplay area, the following Section III will now describe in detail someoperations that the user can perform to composite media clips in thedisplay area.

III. Compositing and Viewing in Display Area

The above sections describe the use of the free-form display area forviewing and organizing media clips. In addition to these organizingoperations, some embodiments provide the functionality for a user tocomposite media clips in the free-form display area in order to create amedia presentation. A user can create sequences of media clips withinthe display area in some embodiments by concatenating a first media clipto a second media clip. As mentioned above, a sequence of media clips isa group of media clips associated in a particular chronological order.

A. Concatenating Clips to Form a Sequence

For some embodiments of the invention, FIG. 28 illustrates an example ofthe creation of sequences of media clips in a free-form display area700. Specifically, FIG. 28 illustrates the creation of two sequences ofmedia clips in the display area 700 by concatenating the media clipsover six stages 2860-2885. At each of the above stages, a media clip isconcatenated to one of the sequences. As shown, the display area 700displays media clips 2820-2850 as well as other media clips.

The display area 700 also allows the user to create sequences of mediaclips by concatenating multiple media clips horizontally. At the firststage 2860, the media clips 2820-2850 are all displayed individually, asno clips are associated with any other clips. The second stage 2865illustrates that a user has dragged a clip 2825 such that the left edgeof 2825 is next to the right edge of 2820. This creates a sequence 2890of these two clips. Some embodiments create a data structure for thesequence, which is described below by reference to FIG. 29. The twoclips can now be moved as one entity, and can be played back as though asingle video clip in some embodiments.

In order to concatenate the two media clips together, some embodimentsdefine an area (having a particular size) around the edge of anon-moving first clip. When a second clip is moved into this area (and,in some embodiments, if the movement has a particular set ofproperties), the edge of the moving second clip is automatically snappedinto alignment with the edge of the first clip. This snapping process isdescribed in further detail below by respect to FIGS. 31 and 30.

The third stage 2870 illustrates that a user has added another mediaclip 2830 on the right edge of the sequence 2890. The resulting sequenceis sequence 2891. Thus, when a user plays back the sequence 2891, mediaclip 2820 will play, followed by clip 2825, and then clip 2830. Someembodiments do not impose any limit on the number of clips that can beconcatenated into a sequence.

At the fourth stage 2875, the user has dragged a media clip 2850 to theright edge of media clip 2845, thereby creating a second sequence 2895.The free-form display area 700 now includes two distinct sequences thatare separate from each other. Some embodiments allow a user to create asmany sequences as the user wants to in the free-form display area.

The fifth stage 2880 illustrates that the user has moved a media clip2840 to the left edge of the media clip 2845, thereby adding the mediaclip 2840 to the beginning of the sequence 2895. As shown, the rightedge of a moving clip can be snapped to the left edge of a non-movingclip in some embodiments. When a user plays back the second sequence2895, media clip 2840 will play, followed by clip 2845, and then clip2850.

The sixth stage 2885 illustrates that the user has moved a media clip2835 to the right edge of the first sequence 2891, such that thissequence now includes four clips. In some embodiments, users can moveback and forth between various sequences, adding clips to a firstsequence, then a second, then back to the first sequence.

In some embodiments, each of the clips is moved separately. Forinstance, when a user controls the media-editing application with acursor controller such as a mouse, each media clip is selected anddragged to its new location in a separate interaction. On the otherhand, when the media-editing application operates on a touch-screendevice (e.g., an iPad®), some embodiments allow a user to move multipleclips at once with multiple fingers. For example, a coordinated usercould create sequence 2890 with the left hand and sequence 2895 with theright hand in one fluid motion.

FIG. 29 illustrates a data structure 2900 for a sequence according tosome embodiments. The media-editing application creates a data structuresuch as structure 2900 when two clips are concatenated to begin theformation of a sequence. The sequence data structure 2900 includesvarious pieces of information that defines the unique sequence. The datastructure 2900 includes a sequence ID field that uniquely identifies thesequence, and a list of the clips that form the sequence. Each clip isassigned a position in the sequence (e.g., 1, 2, 3, etc.) that definesthe order of the media clips in the sequence. When a clip is added tothe beginning of the sequence, information about the new clip may beadded to the end of the data structure. The new clip is assignedposition 1, and the positions of the other clips in the sequence areupdated.

Each clip listing is a reference to a data structure for the particularmedia clip in some embodiments. As shown, the reference for clip 1 linksto a data structure 2905 for the media clip. As described above, themedia clip data structure of some embodiments includes a reference to asource media file (e.g., an audio or video file) as well as in andout-points in the source file.

Different embodiments may include different information in a datastructure for a sequence. For instance, some embodiments might include acount of the number of clips in the sequence. Some embodiments includelocation information for the sequence in the free-form display area. Asdescribed above, some embodiments of the free-form display area includethis location information in the data structure for the display area.However, in other embodiments, this information is stored in the datastructure for the clips and/or sequences. Yet other embodiments storethis information the data structure for the display area as well as forthe clips and/or sequences.

As described above, some embodiments concatenate two clips together whena first clip is moved within a threshold area of a second clip. FIG. 30conceptually illustrates a process 3000 of some embodiments for snappinga first clip to a second clip in a free-form display area. The process3000 will be described by reference to FIG. 31.

FIG. 31 illustrates an example of the operation of the snapping featureof the free-form display area of some embodiments. FIG. 31 illustratesmedia clips 2820-2835 from the display area 700 of FIG. 28. Forsimplicity, the full display area 700 is not illustrated in FIG. 31.FIG. 31 illustrates in detail the stage 2885 at which media clip 2835 isadded to the sequence 2891, in three sub-stages 3110-3130.

As shown, the process 3000 begins by identifying (at 3005) a movingfirst media clip. The media clip may be moving as a result of input froma cursor controller, through a touch screen, etc. As mentioned above, insome embodiments, multiple clips may be moving at once (e.g., when themedia-editing application operates on a touch-screen device). In thiscase, multiple instantiations of the process 3000 may be performed atonce, or a separate process for keeping track of the multiple movingclips may be performed.

The process then determines (at 3010) whether the moving media clip iswithin a threshold distance of any other media clips in the free-formdisplay area. In some embodiments, as described above, each of the mediaclips is represented as a uniformly-dimensioned rectangle in thefree-form display area. In such embodiments, the left edge of a firstclip and the right edge of a second clip are the same height and areparallel.

Some embodiments determine whether the moving clip is within a thresholddistance of a second media clip by determining whether a particularpoint on either the right or left edge (e.g., the top corners) of themoving clip is within a threshold distance of a corresponding point oneither the left or right edge of a non-moving second clip. Someembodiments define magnet points at the upper corners of each clip, anda right edge magnet point can only be attracted to a left edge magnetpoint of another clip, and vice versa. Other embodiments define amagnetized area along the entire edge of the static clip, such that ifthe top left corner of a moving clip comes within a threshold of anypoint on the right edge of the static clip (or vice versa), the clipsmay be snapped together.

Stage 3110 of FIG. 31 illustrates that the media clip 2830 has a magnetpoint 3105 at its top right corner, and the moving media clip 2835 has amagnet point 3115 at its top left corner. At this stage, the media clip2835 is moving towards the static media clip 2830, but is not yet withinthe threshold distance of the static media clip. Stage 3120 illustratesthat the media clip 2835 is now within the threshold distance of thestatic media clip 2830, as its magnet point 3115 is within the range ofmagnet point 3105.

Returning to FIG. 30, when the process 3000 determines that the movingclip is not within the threshold distance of another clip, the processproceeds to 3030, described below. However, when the moving first clipis within the threshold of a second clip, the process identifies (at3015) this second clip.

The process then determines (at 3020) whether the movement of the firstclip satisfies any requirements for snapping the first clip to thesecond clip. Some embodiments evaluate the velocity (both speed anddirection of movement) and acceleration of the media clip to determinewhether the user intends to concatenate the moving first clip to thestatic second clip. For instance, when a user intends to concatenate theclips, the user will often slow down (decelerate) the first clip as itnears the second clip. On the other hand, if the clip is still movingfairly quickly, the user probably intends to move the first clip pastthe second clip to a new location in the free-form display area.

When the movement of the first clip satisfies the snapping requirements,the process snaps (at 3025) the moving first clip to the non-movingsecond clip, and then ends. Some embodiments snap the first clip to thesecond clip by aligning the upper left corner of the first clip with theupper right corner of the second clip (or vice versa). The non-movingclip stays in the same location in the display area, and the moving clipis positioned so as to align with the non-moving clip. Stage 3130illustrates that as a result of being within the threshold distance ofclip 2830, clip 2835 has snapped to clip 2830, and is added onto thesequence 2891.

B. Inserting a Clip into a Sequence

In some embodiments, users can also insert a media clip into the middleof a sequence between two other clips in the sequence. FIG. 32illustrates an example of moving a clip 3205 into the middle of asequence 3225 that contains media clips 3210, 3215, and 3220 within afree-form display area 700. FIG. 32 illustrates this process in fourstages 3230-3260.

In the first stage 3230, the user drags media clip 3205 from a locationin the display area indicated by the dotted media clip 3205 to a newlocation over media clips 3210 and 3215 as shown. In some embodiments,simply moving the individual clip to this location will notautomatically insert the clip into the sequence. For example, someembodiments also allow a user to pile a media clip (or a sequence) ontop of a sequence, without inserting the media clip into the sequence.

The second stage 3240 illustrates that the user decides to insert mediaclip 3205 between media clips 3210 and 3215 in the sequence 3225. Asshown, the user moves media clip 3205 slightly downward in the displayarea 700 in order to insert the media clip into the sequence. Differentembodiments recognize different inputs for inserting the clip into thesequence. Various examples of such inputs include hotkeys, holding themedia clip 3205 in approximately the same position over the two clips3210 and 3215, etc. Some embodiments also require that the media clipbeing inserted (clip 3205) cover at least a particular portion (e.g.,half) of the edge between the two adjacent media clips (clips 3210 and3215) in order to insert the clip into the sequence. Other embodimentsdo not differentiate whether the media clip 3205 is inserted from thetop of the sequence 3220 as shown or from the bottom of the sequence aslong as the clip being inserted is placed between the two media clips ofthe sequence 3220.

The third stage 3250 illustrates that media clips 3215 and 3220 (allclips in the sequence after the insertion point) have slid to the rightto make a room for media clip 3205 in the sequence 3225. In someembodiments, the media-editing application snaps the inserted media clipinto the space created between the formerly adjacent media clips. Inthis example, media clip 3205 snaps into the space that media clip 3215used to occupy. The fourth stage illustrates the resulting sequence 3226after media clip 3215 has been inserted into the sequence 3225.

As described above, when a user inserts a media clip into the middle ofa sequence, the data structure for the sequence is updated to includethe new media clip. Some embodiments add the new media clip at the endof a list of media clips and update the position information for anyrequired media clips in the clip listing of the sequence data structure.

C. Removing a Clip from a Sequence

In addition to creating sequences by adding clips together, a user maywant to remove a clip from a sequence. Some embodiments allow a user toremove a clip either from the middle of a sequence or from the end of asequence. FIG. 33 illustrates an example of removing a media clip 3205from the middle of a sequence 3226 that contains clips 3205-3220 withina free-form display area 700. FIG. 33 illustrates this process in fourstages 3328-3350.

At the first stage 3328, the user has selected media clip 3205 in thesequence 3226. In some embodiments, the media clip 3205 is selected by auser clicking on the media clip, or double-clicking (or double-tapping)in the case where a single click (or tap) selects the sequence of whichthe media clip is a part.

At the second stage 3330, the user has moved the cursor (or, if using atouch screen, the user's finger) down and to the right in order to movemedia clip 3205 out of the space in the sequence that media clip 3205 isoccupying in the direction indicated by the dotted arrow. In someembodiments, when a media clip in a sequence is moved out of a space inthe sequence that the media clip has occupied and leaves a substantialportion (e.g., more than half) of the space empty, one of the possibletwo media clips and other media clips adjacent those media clips slidein so that the space that used to be occupied by the moving out mediaclip does not remain unoccupied after the media clip moves out.

The third stage 3340 illustrates that media clip 3205 has been moved outof the sequence 3226. Some embodiments require that the user input atleast a minimum threshold amount of movement in order to break the mediaclip free of the sequence (e.g., at least a particular distance or atleast a particular speed). In this case, the user has moved the cursordown and to the right fast enough to break the media clip 3205 free ofthe sequence 3205.

When the media clip is moved out of the sequence, any clips after themedia clip in the sequence are moved to the left so that the sequencestays together. As shown at the fourth stage 3350, the clips 3215 and3220 are moved to the left so that clip 3215 is adjacent to clip 3210.The media clip 3205 is now on its own in the display area 700.

FIG. 34 illustrates an example of removing a media clip 3420 from theend of a sequence 3425 that contains clips 3405-3420 within a free-formdisplay area 700. This process is illustrated in two stages 3430 and3440.

In the first stage 3430, the user selects media clip 3420 to remove itfrom the sequence. In some embodiments, the requirements for removing amedia clip from the end of a sequence are the same as for removing aclip from the middle of a sequence (e.g., moving a cursor a far enoughdistance at a fast enough speed). The second stage 3440 illustrates thatthe sequence 3425 remains without the media clip 3420, which is now onits own in the display area 700. In this example, media clip 3420 isremoved from the right side of the sequence (i.e., media clip 3420 isthe last media clip in the sequence). It is also possible to remove thefirst media clip of the sequence in a similar manner.

When a media clip is removed from a sequence, some embodiments removethe reference to the media clip in the data structure for the sequence.In addition, when the media clip is removed from the beginning or middleof the sequence, the position information of the other clips listed inthe sequence data structure is adjusted.

FIGS. 32-34 described above illustrate examples of editing a sequence byinserting a media clip into the sequence, removing a media clip from thesequence, augment the sequence with a media clip or another sequence.These editing operations illustrated in FIGS. 32-33 can be used inconjunction or in succession in order to edit a sequence by reorderingthe media clips in the sequence. For instance, media clip 3205 removedfrom the sequence illustrated in FIG. 33 can be inserted between mediaclips 3215 and 3220 or added to the sequence by snapping media clip 3205to media clip 3210 or 3220.

The above subsections described the creation of sequences throughvarious media-editing operations. The following subsections willdescribe various operations performed on the sequences, such as movingsequences, piling sequences, concatenating sequences, stackingsequences, collapsing sequences, etc.

D. Moving Sequences

FIG. 35 illustrates an example of the movement of a sequence to a newlocation within a free-form display area 700. FIG. 35 illustrates thedisplay area 700 at two different stages 3520 and 3530. The display area700 displays media clips 3530 and 3535, which form a sequence 3540,along with other media clips and sequences. The several sequencesillustrated in FIG. 35, including sequence 3540, are created by the userin a similar manner to that described above by reference to FIG. 28.

The display area 700 also allows a user to move sequences anywherewithin the display area in order to organize and composite the mediaclips and sequences. In some embodiments, the user moves these sequencesto new locations within the display area 700 in a similar manner to thatdescribed above by reference to FIG. 15.

In the first stage 3520, the user selects the sequence 3540 and movesthe sequence up and to the right within the free-form display area.Different embodiments enable different ways for the user to select asequence. In some embodiments, when clips are concatenated in asequence, selecting (e.g., clicking on, tapping) a clip automaticallyselects the entire sequence. In some embodiments, the default selectionis for a single clip and double-clicking (or double-tapping) the clipwill select the entire sequence. The user can select a sequence byenclosing the entire sequence in a selection box or selecting each clipin the sequence (e.g., while holding down a shift key) in someembodiments. The user then performs a drag-and-drop operation on theselected sequence in order to place the sequence in a desired locationwithin the display area. As mentioned, stage 3520 illustrates the usermoving the sequence 3540 up and to the right.

The second stage 3530 illustrates that the sequence 3540 has been movedto a new location within the display area 700 next to another sequence3545. The user may have chosen this location in order to see whether theuser wants to combine the two sequences into one sequence. The dashedversion of sequence 3540 indicates that sequence's original locationbefore being moved.

E. Concatenating Sequences

In addition to concatenating a clip to another clip or sequence asdescribed above in subsection A, some embodiments allow a user toconcatenate a first sequence to the beginning or end of a secondsequence. FIG. 36 illustrates this process in two stages 3620 and 3630.

In the first stage 3620, the user selects sequence 3605 and drags thesequence down and right such that the left edge of the first clip insequence 3605 is near the right edge of sequence 3610. In someembodiments, the user selects the sequence 3605 in any of the waysdescribed above in subsection C (e.g., clicking on a clip, enclosing theentire sequence, etc.).

The second stage 3630 illustrates that sequence 3610 has been augmentedby adding the media clips of sequence 3605. In some embodiments, whenthe left edge of sequence 3605 comes within a threshold distance of theright edge of sequence 3610 (i.e., the right edge of the right-mostmedia clip in the sequence), the media-editing application concatenatessequence 3605 into sequence 3610 in a similar manner as with the twomedia clips described above in detail by reference to FIG. 31. In thisexample, sequence 3605 is added to the right side of sequence 3610. Itis also possible to add a first sequence to the left side of a secondsequence in a similar manner.

When the user adds the sequence 3605 to the end of sequence 3610, someembodiments update the data structure for sequence 3610 to include allof the media clips from sequence 3605. In addition, some embodimentsdelete the data structure for sequence 3605, as it is no longer anindependent sequence. Other embodiments update the data structure forthe moved sequence 3605 and delete the data structure for the staticsequence 3610.

F. Piling Sequences

FIG. 37 illustrates an example of the piling of sequences in a free-formdisplay area 700. The display area 700 displays sequences 3720 and 3725,along with other sequences and individual media clips. The severalsequences illustrated in FIG. 37, including sequences 3720 and 3725, arecreated by the user in a similar manner to that described above byreference to FIG. 28.

The display area 700 also allows a user to pile up sequences of mediaclips within the display area. In some embodiments, the user piles upthe sequences in a similar manner to that described above by referenceto FIG. 16 for single clips. As shown in FIG. 37, the user has selectedand moved a sequence 3725 (e.g., with a drag-and-drop operation) similarto the selection and movement of sequence 3540. In this case, thesequence 3725 is piled on top of the sequence 3720. This allows the userto create a visual grouping of these two sequences. Some embodimentsalso create a logical grouping in the data structures of themedia-editing application between the two sequences and/or between allof the clips in the sequences. For instance, some embodiments append atag referencing the data structure for sequence 3725 in the datastructure for sequence 3720.

G. Stacking Clips and Sequences

In addition to snapping clips together horizontally to build sequences,some embodiments snap clips and sequences together vertically. Thisallows users to easily move pairs (or larger groups) of clips andsequences together without having to composite the clips together assequences. It can also be used to associate two clips or sequenceswithout compositing the clips as a sequence. In addition, it can also beused to layer the media content represented by the top clip over themedia content represented by the bottom clip in a composite mediapresentation. FIG. 38 illustrates an example of stacking two clips 3815and 3825 together in a free-form display area 700. FIG. 38 illustratesthis process in five stages 3810-3850.

In the first stage 3810, a user drags the media clip 3815 down and tothe right, towards the non-moving media clip 3825. As shown at thisstage, media clip 3825 includes a set of magnets along its top edge andmedia clip 3815 includes a set of “oppositely polarized” magnets alongits bottom edge. Thus, when the bottom of a moving first media clip isbrought within a threshold distance of the top of a non-moving secondmedia clip, the two clips are snapped together in some embodiments.Similarly, when the top of a moving first media clip is brought within athreshold distance of the bottom of a non-moving media clip, the twoclips are also snapped together.

The second stage 3820 illustrates that the bottom of the moving clip3815 is within the magnetic threshold of non-moving clip 3825. As shown,some embodiments snap the moving clip straight down and do notautomatically align the clips horizontally. Because the display area istimeline-free, there is not necessarily any inherent meaning to havingone clip vertically aligned with another. However, some embodimentscreate layers of media content with the media content represented by thetop clip as a top layer and the media content represented by the bottomclip as a bottom layer. In addition, as described above with respect tohorizontal snapping, some embodiments impose requirements on the clipmovement in order to snap the clips together. That is, based on theclip's velocity and acceleration (and, in some cases, other factors),the media-editing application determines whether to snap the clipstogether.

The third stage 3830 illustrates that the moved clip 3815 has beensnapped straight down to connect with the clip 3825. In addition, atthis stage, after the clips have snapped together, the user (with theclip 3815 still selected) has moved the cursor to the right and slightlyupwards. As shown at stage 3840, the clip 3815 slides horizontally alongthe top of clip 3825. In some embodiments, when two clips are stackedvertically, cursor movements (or, if displayed on a touch screen, fingermovements) with less than a threshold amount of vertical movement willbe treated as purely horizontal for the purpose of moving the selectedclip. The fifth stage 3850 illustrates that the clip 3815 has been movedsuch that it is horizontally aligned with the clip 3825. When twostacked clips are within a small threshold of horizontal alignment, someembodiments automatically align the two clips.

As mentioned, some embodiments also stack sequences in a similarfashion. FIG. 39 illustrates an example of stacking two sequences 3905and 3915 together in a free-form display area 700. FIG. 39 illustratesthis process in three stages 3910-3930.

In the first stage 3910, the user drags a selected sequence 3905 downand left towards sequence 3915. Sequence 3905 might be selected in anyof the ways described above (e.g., single or double-clicking, single- ordouble-tapping, enclosing the sequence with a selection box, etc.). Whenthe bottom edge of sequence 3905 is brought within a threshold distanceof the top edge of 3915, the sequence 3905 is automatically snappeddownwards and stacked on top of sequence 3915, as shown at the secondstage 3920. The third stage 3930 illustrates the result, with sequence3905 stacked on top of sequence 3915. As with the clips, sequences canalso be moved so as to snap to the bottom of another sequence or mediaclip.

In some embodiments, an option is provided to display the thumbnails ashaving varying lengths based on the length of the represented mediaclips. For example, a two minute clip would be twice as long as a oneminute clip. In such embodiments, the stacking of sequences can be usedto simulate media tracks, as the displayed length of the sequence wouldbe representative of the temporal length of the sequence.

Some embodiments allow sequences created in the free-form display area(e.g., display area 700) to be dragged or otherwise moved (e.g., via ahotkey) into another compositing space of the GUI, such as the compositedisplay area described above (e.g., by reference to FIG. 11). When asequence is moved into the composite display area, the sequence isplaced on a track. In some embodiments, a user can move a pair ofstacked sequences such that the sequences are placed on two tracks inthe composite display area.

Some embodiments allow the user to layer media content of a sequenceover media content of another sequence in a composite presentationcomposed of these sequences. In some such embodiments, media content inthe top layer is played back over the media content in a bottom layerwhen a composite presentation composed of these sequences is playedback.

The previous two figures illustrated some examples of stacking one mediaclip over another media clip and stacking one sequence over anothersequence. In some embodiments, a media clip can be stacked over asequence, or vice versa, in a similar manner as described above byreference to these two figures.

Moreover, media clips and sequences can be stacked in more than two rowsin some embodiments. Each row of these stacked media clips and sequencesin some embodiments is a layer of the media content represented by themedia clips or sequences of the row in a composite presentation composedof these media clips and sequences.

H. Collapsing Sequences

FIG. 40 illustrates an example of the minimizing (or collapsing) of asequence 4035 into a smaller sequence or an icon in a free-form displayarea 700. FIG. 40 illustrates the free-form display area at threedifferent stages 4020-4030.

The display area 700 also allows the user to minimize or collapse asequence into a smaller display of the sequence, or an icon. In someembodiments, these minimized sequences are organized along the bottom orright edge of the display area 700. In some embodiments, the sequencesare minimized in a manner similar to that described above for individualclips by reference to FIG. 17.

In the first stage 4020, the user has moved a cursor over a selectedsequence 4035. The sequence 4035 might be selected in any of the waysdescribed above (e.g., single or double-clicking, single- ordouble-tapping, enclosing the sequence with a selection box, etc.). Whenthe cursor moves over the sequence, a selectable minimization item 4045is displayed in the bottom left corner of the selected sequence.

At the second stage 4025, the user has moved the cursor over theselectable item 4045 and selected the item. As a result, at the thirdstage 4030, the sequence 4035 has been minimized into icon 4050 andlocated near the bottom edge of the free-form display area 700.

Some embodiments determine whether the icon is closer to the bottom orright edge and display the small icon for a minimized sequence or clipalong that edge. In some embodiments, a user can minimize a clip orsequence by dragging the item to the edge. In such embodiments, thesmall icon is displayed along the edge to which the user drags the item.In addition, some embodiments enable a user to minimize a clip orsequence with a hotkey. When a user wants to restore a minimizedsequence, the user can select the icon in some embodiments, which willautomatically restore the sequence to full size.

I. Skimming Through a Sequence

In some embodiments, a user can skim (or scrub) through a selectedsequence in a display area by dragging a playhead through the sequence.As the user moves the playhead across a particular video clip in thesequence, the frame of the video clip that corresponds temporally to thelocation of the playhead in the video clip is displayed. A user can usethis feature, for example, to find a particular frame of a video clipthat is part of a sequence.

As mentioned above, a sequence is an association of media clips inchronological order. The associated media clips in some embodiments arevideo clips that are composed of multiple image frames. In someembodiments, the user can perform a skimming operation on a sequence ofmedia clips in the display area by moving a cursor over the media clipsin the sequence. The skimming operation on a sequence is performed in asimilar manner described above for a single clip by reference to FIG.19.

Different embodiments implement a skimming operation differently. Forinstance, the user performs the operation by first selecting a sequenceto skim in the free-form display area. When the sequence is selected, aplayhead appears and the user drags the playhead to move it along thelength of the selected sequence. In some cases, the user places thecursor over a media clip to skim without selecting it first. In thesecases, placing the cursor over the media clip causes a playhead toappear on the sequence at the location along the length of the mediaclip where the cursor is placed.

FIG. 41 illustrates an example of skimming through a sequence in afree-form display area 2305 of GUI 2300. As shown, a GUI 2300 includesthe display area 2305 and a viewing area 2310. The display area 2305displays sequence 4145 as well as other sequences and media clips. FIG.41 illustrates the sequence 4145 at three different stages 4110-4130.

In operation, the user moves cursor 4155 over sequence 4145 in thedisplay area 2305. A playhead 4160, depicted as a bold vertical bar,appears on sequence 4145 at the location of the cursor. As the usermoves the playhead 4160 with cursor 4155 to the right to display theframes in the sequence 4145, the image frames represented by newlocations of the playhead 4160 along the length of sequence 4145 aredisplayed in succession as illustrated in stages 4110-4130. At the firststage 4110, the poster frames (or representative thumbnails) for thefirst, third, and fourth clips in the sequence are displayed, and theframe that is approximately two-thirds of the way through the secondclip is displayed. This frame is also displayed in the viewing area 2310in some embodiments.

The frames are displayed within a particular media clip representationwithin the sequence as shown. When the playhead leaves a media clip inthe sequence, the media clip displays its poster frame or keeps the lastframe displayed before the playhead left. The user may move the playhead4160 in either horizontal direction (i.e., to the left or right) todisplay a desired image frame of sequence 4145. At the second stage4120, the playhead is at the first frame of the third media clip insequence 4145 (which also happens to be the poster frame for the mediaclip), while at stage 4130 the playhead is approximately two-fifths ofthe way through the third clip. In both of these stages, the second cliphas gone back to displaying its poster frame.

The user can use this skimming feature to find a frame in a sequencewhich the user desires to use as a starting point from which to lay themedia content represented by a media clip over the media contentrepresented by the sequence in some embodiments. In some suchembodiments, when a media clip is stacked on top of a sequence to layerthe media content represented by the media clip over the media contentrepresented by the sequence, a playhead appears on the sequenceunderneath a point in the displayed length of the media clip. Theplayhead moves along the displayed length of the sequence with the pointin the displayed length of the media clip as the user slides the mediaclip horizontally along the top of the sequence. Different embodimentsdefine different points along the displayed length of the media clipthat the playhead on the sequence moves with. For instance, the playheadon the sequence moves with the left or right edge (i.e., points thatrepresent the first or last frame, respectively) of the media clip.Other embodiments have the playhead of the sequence move with theposition of a cursor along the displayed length of the media clip. Theuser stops sliding when the user finds a frame of the sequence to use asthe starting point of layering a media content represented by the mediaclip on top of the media content represented by the sequence.

FIG. 42 shows an example of picking a frame of a sequence formed ofseveral media clips from which the media content represented by a mediaclip is layered over. FIG. 42 illustrates this process in four stages4210-4240.

The first stage 4210 illustrates media clips 4225, 4235, 4245, and 4255and other media clips. The media clips 4235, 4245, and 4255 form asequence 4215. The user has stacked the media clip 4225 on top of thesequence 4215 in a similar manner as described above by reference toFIGS. 38 and 39. The media clip 4225 is stacked such that its left edgeis about a quarter way from the left edge of the media clip 4235 in thesequence 4215. The poster frames (or, representative thumbnails) aredisplayed for the media clips.

In the second stage 4220, the user selects (e.g., clicking on) the mediaclip as indicated by the thick border of the media clip. A playhead 4260appears on the sequence 4260 when the media clip 4260 is selected asshown. In some embodiments, the playhead 4260 appears when the mediaclip 4225 is just placed on top of the sequence 4250 in the process ofstacking the media clip on top of the sequence. In this example, theplayhead appears at a position along the length of the sequence that isvertically synchronized with the left edge of the media clip 4250.

The media clip 4235 displays a frame represented by the position of theplayhead along the length of the media clip 4235 as shown. The user isabout to slide the media clip 4250 to the right along the top of thesequence 4215 as indicated by the dotted arrow in this stage.

In the third stage 4230, the user has slid the media clip to the rightsuch that the left edge of the media clip 4225 is about a third way fromthe left edge of the media clip 4245 along the length of the media clip4245 in the sequence 4215. The playhead 4260 has moved together with theleft edge of the media clip 4245 as shown. The media clip 4245 displaysa frame represented by the position of the playhead along the length ofthe media clip 4245. The user might want this frame of the media clip4245 to be the starting point for the media content represented by themedia clip 4225 to be layered over the media content represented by themedia clip 4225.

The fourth stage 4240 illustrates that the user has unselected (e.g.,releasing the mouse button and removing the cursor out from) the mediaclip 4225. The media clip 4225 is stacked on top of the sequence 4215from the point along the length of the sequence where the playhead 4260was placed in the third stage. As such, the user has chosen a frame inthe sequence from which the media content represented by the media clip4225 is layered over the content represented by the sequence 4215.

In the embodiments that synchronize the position of a playhead with thehorizontal position of a cursor, as the cursor 4250 is dragging themedia clip 4225 and sliding the media clip to the right as shown in thesecond stage 4220, the playhead 4260 would appear about a half-way alongthe length of the media clip 4245 such that it is verticallysynchronized with the position of the cursor 4250 along the length ofthe media clip 4225 in the second stage 4220. In addition, in some suchembodiments, another playhead appears on the media clip 4225 and themedia clip displays a frame represented by the position of that playheadalong the length of the media clip 4225. These embodiments allow theuser to layer the frame displayed on the top media clip over a desiredframe displayed on the bottom media clip.

In some embodiments, the media clip 4225 is a media clip in a sequence.That is, a sequence including the media clip 4225 is stacked on top ofthe sequence 4215. By selecting the top sequence and sliding along thetop of the bottom sequence, the user is able to select a frame in thebottom sequence from which the media content represented by the topsequence is layered over the media content represented by the bottomsequence.

As mentioned above, media clips or sequences can be stacked in more thantwo rows in some embodiments. In some such embodiments, when a mediaclip in the top row is selected, playheads appear on the media clips orsequences in the rows below the top row such that the user can view theframes represented by the positions of the playheads along the lengthsof media clips or sequences.

In some embodiments, the media clip 4225 is an audio clip representingaudio data in a source audio file. These embodiments allow the user tolayer the audio data over the media content represented by the sequence4215 starting from a desired frame represented by the position of theplayhead 4260 along the length of the sequence 4215.

J. Playing Back a Sequence

As with a single clip, some embodiments enable a user to play back asequence in a free-form display area. When a sequence of video clips isplayed back in the free-form display area, some embodiments play thefirst clip within the thumbnail for the first clip, then the second clipwithin its thumbnail, and so on. A playhead moves across the sequence asthe clips are being played back. In some embodiments, the playhead movesat different speeds across the different clips in a sequence to accountfor the different lengths of the clips. For instance, if a first clip isone minute long, and a second clip is two minutes long, the playheadwill move twice as fast in the display while playing the first clip aswhile playing the second. Some embodiments also play the sequence in aseparate viewing area as though the sequence is a single clip.

FIG. 43 illustrates an example of playing back a sequence 4345 in afree-form display area 4340 of GUI 4300. As shown, the GUI 4300 includesthe display area 4340, a viewing area 4350, and playback controls4355-4365. The display area 4340 displays sequence 4345 as well as othersequences and media clips. The display area 4340 is similar to thefree-form display area 2305 described above by reference to FIG. 23,while the viewing area 4350 is similar to the viewing area 2310described above by reference to FIG. 23.

The playback controls 4355-4365 are the same as the set of playbackcontrols 245-255 described above by reference to FIG. 2. These controlsenable a user to play, pause, or rewind a selected media clip orsequence of media clips. In some embodiments, a set of drop-down menuitems may replace or be used in conjunction with the playback controls4355-4365. In some embodiments, hotkeys are assigned to the playbackcontrol commands such that a user's pressing the hotkey invokes thecorresponding playback control command.

The first stage of FIG. 43 illustrates the sequence shortly after theuser has started playback for the sequence 4345. The playhead is not allthat far into the first clip of the sequence. The current frame isplayed in the clip representation within the sequence, as well as withinthe viewing area 4350. In the following stage 4320, the playhead hasprogressed to approximately one-fifth of the way through the second clipin the sequence 4345. At this point, the first clip displays its posterframe (representative thumbnail), though some embodiments display thelast frame of the clip instead. At the third stage 4330, the playhead isapproximately two-fifths through the third clip, and the poster frame isnow displayed for the second clip.

As mentioned, the viewing area 4350 displays the currently playing videoclip in the sequence. In some embodiments, multiple sequences and/orindividual media clips can be played concurrently in the free-formdisplay area. In some such embodiments, the viewing area 4350 plays thesequence or clip most recently selected for playback by the user. Ifthis sequence or clip ends while another sequence or clip is stillplaying, then the viewing area of some embodiments starts to play thatsequence or clip. FIG. 44 illustrates the full GUI 4300 during thelatter two stages of FIG. 43. As can be seen, the viewing area 4350 ineach of these stages displays the currently playing frame from thesequence 4345.

In some embodiments, a clip or sequence can also be played back in fullscreen mode. Once the sequence is being played back in the display area,the user can invoke the full screen mode by a hotkey, a user interfaceitem, etc. FIG. 45 illustrates an example of playing back a sequence4535 in such a full screen mode in a GUI 4500. FIG. 45 illustrates theplayback of the sequence at five stages 4510-4550. As shown, the GUI4400 includes a display area 4505, a viewing area 4515, and playbackcontrols 4555-4565. The display area 4505 displays a sequence 4535 aswell as other sequences and individual media clips. The controls,display area, and viewing area are the same as shown above in FIG. 43,except that the viewing area 4515 also includes a full screen invocationcontrol 4525. In some embodiments this control is located instead nearthe playback controls 4555-4565.

At the first stage of FIG. 45, the user has invoked the playback featureand the sequence 4535 has begun to play in the display area 4505 and theviewing area 4515. At the second stage, the user places a cursor overthe full screen invocation control 4525 as the sequence continues toplay and selects the control. In some embodiments, a hotkey is used toinvoke the full screen playback mode. As a result, at the third stage4530, the entire GUI 4500 is occupied by the playing video clip. Thestages 4540 and 4550 illustrate that the sequence continues to play backin full screen mode.

As shown, in some embodiments a media clip or sequence can be playedback in full screen mode when the sequence is already being played backin the display area. In some embodiments, the user can first select asequence to play back and switch to the full screen mode before the userstarts to play back the sequence. In some such embodiments, the posterframe of the left-most media clip in the sequence is displayed in fullscreen mode and then the user can invoke a playback command. A playheaddoes not appear in the full screen mode in some embodiments.

In some embodiments, many of the editing and organizing featuresdescribed in the previous sub-sections can be performed as the sequenceis being played back in the free-form display area. For example, userscan concatenate additional media clips to a sequence, remove media clipsfrom a sequence, move a sequence in the display area, pile or stacksequences, etc., while one or more of the sequences is playing. Inaddition, other editing operations on both single clips and sequences,described below, may also be performed as the video is played back inthe free-form display area.

IV. Editing Media Clips in Display Area

As mentioned above, some embodiments allow a user of a media-editingapplication to edit media clips in a free-form display area of themedia-editing application. The user can create composite presentationsin the display area by concatenating multiple media clips intosequences, as described in the previous section. The user also can editmedia clips in the display area in other ways, as will be described indetail below. Sub-section IV.A will describe several embodiments thatallow the user to edit single media clips, while sub-section IV.B willthen describe several examples of editing composite presentations.

A. Editing Single Media Clips

Some embodiments provide several different operations for a user of amedia-editing application to perform to edit single media clips in adisplay area. Some such operations include marking regions of interestwithin media clips, duplicating media clips, tagging and filtering mediaclips, trimming media clips and other editing commands. Beforedescribing these editing operations, the following subsection willdescribe a feature of some embodiments that displays the file namesassociated with media clips in the free-form display area. Thesubsections following this will then describe various editing operationsthat a user can perform on media clips in the free-form display area.

i. Revealing Names of Media Clips

As mentioned, some embodiments provide an option that enables a user tocause the media-editing application to display file names associatedwith media clips in the free-form display area. As described above, eachmedia clip in the free-form display area refers to a source file. Thefile name of this source file is displayed over the media clips in someembodiments.

FIG. 46 illustrates an example of the use of such a name display featurein a GUI 4600. FIG. 46 illustrates the operation of this feature in twostages 4620 and 4630. The GUI 4600 includes a display area 4610 that issimilar to the free-form display area 700 described above by referenceto FIG. 7, and also includes a selectable name display UI item 4605. Theselection of the UI item 4605 causes the names of the source media fileswith which the media clips in the free-form display area are associatedto be displayed.

At the first stage 4620, a user has moved a cursor over the selectableitem 4605. In various embodiments, the name display feature may beinvoked in various ways: through a drop-down menu, a hotkey, etc. At thesecond stage 4630, the names of the media files for the media clips areshown over the bottom of the thumbnail representing the media clips. Inthis example, names are displayed for all of the media clips in thefree-form display area. Some embodiments allow a user to selectivelydisplay names for media clips by selecting particular clips and theninvoking the name display feature. In some embodiments, a user caninvoke the command a second time to have the names no longer displayed.

Different embodiments display the names of media clips differently. Forinstance, the names can be displayed inside the media clip as shown.Some embodiments enable a user to configure the location in or aroundthe clip representation where the file names are displayed (e.g., at thebottom, at the top, underneath, etc.).

ii. Regions of Interest

Some embodiments enable a user to mark a region of interest of a mediaclip in the free-form display area. A region of interest is a temporalportion of a media clip that is selected by the user. FIG. 47conceptually illustrates a process 4700 of some embodiments for markinga region of interest of a video clip. The process will be described byreference to FIGS. 48 and 49, which illustrate different ways to mark aregion of interest in a media clip.

Each of FIGS. 48 and 49 illustrates a media clip 4800 over three stages.In FIG. 48, the user selects a region of interest while scrubbingthrough the media clip 4800 over stages 4810-4830. In FIG. 49, the userselects a region of interest while playing back the media clip 4800 overthree stages 4910-4930. Stage 4810 illustrates that a user is scrubbingthrough the media clip by dragging the playhead with a cursor. Stage4910 illustrates that the user has begun playback of the media clip.

Returning to FIG. 47, the process 4700 begins by receiving (at 4710) abeginning point in a media clip for a region of interest. The beginningpoint may be anywhere in the temporal length of the media clip. Stage4820 illustrates that the playhead has been moved to a first location inthe media clip (approximately two-fifths of the way through). At thisjuncture, the user performs input to indicate the marking of the mediaclip. In some embodiments, the user holds a mouse button down along witha hotkey and drags the playhead across the media clip.

Stage 4920 illustrates that the playhead is approximately two-fifths ofthe way through the media clip in its playback. At this point, the userperforms input to indicate the marking of the media clip. In someembodiments, the user holds down a hotkey during playback or performsother input in order to select the region of interest.

The process 4700 next receives (at 4715) an end point in the media clipfor the region of interest. The end point may be anywhere in thetemporal length of the media clip after the beginning point. Stage 4830illustrates that the playhead has been moved to a second location in themedia clip that is approximately five-sixths of the way through theclip. At this point, the user has released the hotkey and/or mousebutton in order to mark the endpoint. Similarly, at stage 4930illustrates that the user has performed input (e.g., releasing thehotkey) to mark the end of the region of interest during the playback ofthe media clip.

The process then adds (at 4720) the region of interest to the datastructure for the media clip. Some embodiments, as described above,store a data structure for each media clip in the free-form displayarea. The process 4700 adds information about the region of interest(i.e., the beginning and end points in the timecode of the source file)to the data structure for the media clip.

The process also creates (at 4725) a selectable marker over therepresentation of the media clip in the free-form display area. As shownat stage 4830 (and at stage 4930), a marker 4825 (or 4925) is displayedthat spans the selected region of interest. While the markers are shownas a gray bar, one of ordinary skill will recognize that any other userinterface item may be used. The illustrated marker spans a portion ofthe thumbnail representing the media clip that corresponds to thetemporal position of the region of interest.

Once a region of interest is selected, the user can create a new mediaclip that contains only the media spanned by the region of interest(i.e., eliminating the rest of the original media clip). FIG. 50conceptually illustrates a process 5000 of some embodiments forextracting a region of interest to create a new media clip. The process5000 will be described by reference to FIG. 51, which illustrates suchan extraction in a GUI of some embodiments.

FIG. 51 illustrates the extraction of a region of interest 5115 from amedia clip 5120 over two stages, 5130 and 5140. In the first stage 5130,the media clip is displayed in a free-form display area 2305 of the GUI2300.

As shown, the process 5000 begins by receiving (at 5010) a selection ofa marker representing a region of interest of a media clip. The user mayselect the marker by touching it on a touch screen, clicking a mousebutton with the cursor over the marker, etc. The process then determines(at 5015) whether the marker has been dragged out of the media clip. Inorder to create a new clip in some embodiments, the user selects amarker and drags it outside of the clip.

Thus, when the marker has been dragged out of the media clip, theprocess creates (at 5025) a new clip from the region of interest andends. The new clip that is created will refer to the same source mediafile as the clip from which it was created, but will have different inand out-points. The in and out-points of the new clip will be the startand end points of the region of interest in some embodiments. In someembodiments, rather than extracting the selected region of interest bydragging, a user can press a hotkey or select an option from a drop-downmenu in order to create a new media clip using the selection region. Forinstance, by pressing a hotkey or selection an option from a drop-downmenu, the user can copy the marker and paste in a display area to createa new media clip.

The second stage 5140 illustrates that the user has moved the region ofinterest outside of the media clip 5120 to create a new clip 5125. Asillustrated in the exploded view, the start of clip 5125 is the start ofthe region of interest 5115, and the end of clip 5125 is the end of theregion of interest 5115.

When the marker has not been dragged out of its media clip, the process5000 determines (at 5020) whether the marker is still selected. If auser has unselected the region of interest (e.g., if the user decidesnot to extract the region of interest) or deleted the region of interest(in some embodiments, the region of interest is deletable), the processends. Otherwise, the process returns to 5015 to again determine whetherthe region of interest has been extracted.

Regions of interest of a media clip can also be adjusted in someembodiments. Such regions can be expanded, shrunk, or moved in someembodiments. FIG. 52 illustrates two scenarios 5200 and 5210, each ofwhich illustrates a media clip 5205 in two stages. The media clip 5205includes a region of interest represented by a marker 5215.

In the first scenario 5200, a user shrinks the region of interest. Theuser places a cursor over the right edge of the marker 5215, selectsthis edge (e.g., by pressing down on a mouse button), and moves thecursor to the left. Some embodiments define a portion of the region ofinterest marker 5215 over which the cursor can be when the mouse buttonis pressed in order to select the edge for adjustment. By moving thecursor to the left with the right (ending) edge selected, the end pointof the region of interest is moved earlier in the media clip, therebyshortening the region of interest. The resulting marker is marker 5216.The user can also select the left (starting) edge and move it to theright in order to shorten the region of interest by moving the startpoint later in the media clip.

In the second scenario 5210, a user expands the region of interest. Theuser places a cursor over the left edge of the marker 5216, selects thisedge (e.g., by pressing down on a mouse button), and moves the cursor tothe left. By moving the cursor to the left with the left (starting) edgeselected, the start point of the region of interest is moved earlier inthe media clip, thereby expanding the region of interest. The resultingmarker is marker 5217. The user can also select the right (ending) edgeand move it to the right in order to expand the region of interest bymoving the end point later in the media clip.

Some embodiments also allow a user to keep the length of the region ofinterest the same, but move the region of interest within the mediaclip. By selecting a portion of the region of interest close to thecenter (i.e., away from the edges) and moving the cursor, the region ofinterest can slide along the timeline of the media clip.

iii. Duplication of Media Clips

Within the free-form display area of some embodiments, duplicates ofmultiple clips can exist. When clips are duplicates, some embodimentscreate duplicate data structures that point to the same source mediafile and have the same in and out-points. A user might want duplicateclips so that the user can insert the clip into different sequences tosee how the different sequences will look. In addition, a user can editone of the duplicates to modify the in and out-points and createdifferent media clips that point to the same source media file.

FIG. 53 illustrates examples of duplicating media clip in a free-formdisplay area 5305 of some embodiments. FIG. 53 illustrates the operationof this feature in four stages of a GUI 5300: in the first two stages5310 and 5320, the same clip is inserted twice from a browser 5315,while in the third and fourth stage 5330 and 5340 a clip is duplicatedwithin the free-form display area. This figure illustrates that thefree-form display area 5305 and the browser 5315 are part of the sameGUI 5300 for a media-editing application. In some embodiments, thebrowser operates separately from the media-editing application (e.g.,the browser could be a file folder).

As shown at stage 5310, the user has selected a source media file(“concert”) in the browser 5315 and dragged that into the free-formdisplay area 5305 to create a media clip 5325. At this point, someembodiments create a data structure for the media clip. The in andout-points of this newly created clip are the start and end timecodes ofthe source file in some embodiments. At the second stage 5320, the userhas again selected the same source media file (“concert”) in the browser5315 and dragged that into a different location in the free-form displayarea 5305 to create another media clip 5335. The media clip 5335 willhave the same information in its data structure as that for clip 5325,except that its unique ID will be different.

Stage 5330 illustrates that a user has selected a media clip 5345 andexecuted a duplicate clip command. In some embodiments, the user canright click on the clip to cause a drop-down menu to appear, and one ofthe commands in the menu is a duplicate clip command. In someembodiments, the user can also use a hotkey to cause the menu to appear,or use a hotkey to execute the command. At stage 5340, a new media clip5355 has appeared in the free-form display area that is a duplicate ofthe selected clip 5345. A new data structure is created for this clipthat has the same information (source file, in and out-points) as thedata structure for clip 5345, except with a different unique identifier.In some embodiments, a user can also create a region of interest that isthe entire length of a media clip and then extract that region to arriveat the same result.

iv. Tagging and Filtering Media Clips

In some embodiments, media clips in the free-form display area can betagged and thereby grouped into categories. Users can type in tags forthe media clips, select from a group of listed tags, etc. Once a list ofone or more tags exists for a project, some embodiments performauto-completion as a user types in a tag. In addition, users can filterthe display of media clips in the free-form display area based on tags.Users can select a tag and the media-editing application will displayonly clips that are tagged with the selected tag in the free-formdisplay area.

Some embodiments perform the tagging at the frame level rather than theclip level, or at both levels. A user can tag a specific frame or set offrames in some embodiments, and this tag applies to both the frame andthe clip. If the frame is extracted as part of a new clip, the clip willalso have the same tag in some embodiments.

FIG. 54 illustrates an example of tagging a media clip in a free-formdisplay area 5405 of some embodiments. FIG. 54 illustrates this taggingprocess in four stages 5410-5440 of a GUI 5400 that includes the displayarea 5405. The display area 5405 is similar to the free-form displayarea 700 described above by reference to FIG. 7. The display area 5405includes a tag input UI item 5415 and a media clip 5425, as well asseveral other media clips.

In the first stage 5410, a playhead 5435 is currently at a particularframe early in clip 5425. The user could have scrubbed through the mediaclip to that frame or the media clip could be playing back. At thesecond stage 5420, the user has selected the tag input UI item 5415 witha cursor. In some embodiments, the user accesses the taggingfunctionality through a hotkey, a menu option, etc. As a result of theselection of the UI item 5415, a text input window is now displayedbelow the media clip 5425.

At the third stage 5430, the user has typed the word “bike” into thetext input window below the media clip 5425. The fourth stage 5440illustrates that the user has input “bike” as a tag for the media clip5425. At least the frame at which the playhead was located at stage 5420is tagged with the word “bike”. The location of this frame in thetemporal length of the media clip is indicated by the marker in theclip. In addition, a bin 5445 for the tag “bike” has been created in abrowser 5450.

Some embodiments store the tag information as metadata about the mediaclip. In some embodiments, the tag and the frames of the media clip withwhich it is associated is stored in the data structure for the mediaclip. The metadata in the data structures is discussed in further detailbelow.

FIG. 55 conceptually illustrates a process 5500 of some embodiments fortagging a media clip. The process 5500 is performed by the media-editingapplication as a user types in a tag for a media clip (or frame of amedia clip), and suggests tags based on what the user has typed and anytags already created for media clips in the project that includes themedia clip. The process 5500 will be described by reference to FIG. 56,which illustrates the GUI 5400 in four stages 5610-5640. At the firststage 5610, the GUI 5400 has two tag bins with one media clip each.Media clip 5425 is tagged with “bike” and therefore has been placed bin5445, and media clip 5625 is tagged with “ride” and therefore has beenplaced in bin 5455.

Returning to FIG. 55, the process 5500 begins by receiving (at 5505) atag command for a media clip. As described above, this command may bereceived by selection of a selectable UI item (as shown in FIG. 54),selection from a drop-down or other menu, typing of a hotkey, etc. Afterreceiving the tag command, the process 5500 identifies (at 5510) theframe position of the media clip. In some embodiments, the frameposition is the current location of the playhead in the selected mediaclip that is being tagged. The playhead could have been scrubbed to itscurrent location or might be at the position as a result of the mediaclip being played back.

Stage 5620 illustrates that a user has selected the tag entry UI item5415 and that a text entry field has appeared underneath the playhead5435, which is approximately two-thirds of the way through media clip5425. At this point, the operation of the GUI 5400 is similar to itsoperation in the first two stages of FIG. 54.

The process 5500 next receives (at 5515) tag input (e.g., from a usertyping text into the text input field). At stage 5630 of FIG. 56, theuser has typed the letter “r” into the text input field. The processdetermines (at 5520) whether the received input matches at least one binname. That is, the process checks the typed input against the varioustags that have already been input for the project that is displayed inthe free-form display area.

When the input matches one of the tags that has already been created,the process suggests (at 5525) the tag. When multiple tags match (e.g.,the user has typed the letter “b” and the tags “baseball” and“basketball” already exist, different embodiments use different criteriato determine which tag to suggest. Some embodiments go in alphabeticalorder (i.e., suggesting the first tag that matches). Instead of or inconjunction with going in alphabetical order, other embodiments suggestthe tag that has been used the most or that has been used most recently.Stage 5630 of FIG. 56 illustrates that as the user has typed the letter“r”, the rest of the word “ride” appears in light text, indicating thatthe word “ride” is suggested but that the user has not typed the letters“ide”.

The process then determines (at 5530) whether tag input is still beingreceived. The user could be deleting letters, continuing to type, etc.When the user has not finished inputting the tag, the process returns to5520, described above. When the user has finished inputting the tag, theprocess determines (at 5535) whether a bin already exists for the inputtag.

In the case that a user chooses to use a suggested tag, a bin willalready exist for that tag. A user may also type out a full tag name forwhich a bin already exists while ignoring the suggestion feature. On theother hand, in some cases (e.g., the first time a tag is used for aproject) a bin will not yet exist for the input tag. In this case, theprocess 5500 creates (at 5540) a new bin for the input tag. In eithercase, the process then places (at 5545) the tagged media clip in theappropriate bin. Finally, the process 5500 tags (at 5550) the frameidentified at 5510. The process then ends.

Stage 5640 illustrates that the media clip 5435 is now tagged with thetag “ride” and added to the bin 5645 for the tag “ride”. This bin nowhas two tags. Additionally, the media clip 5435 has now been placed inmultiple bins, as it is tagged with both the tag “bike” and the tag“ride”. In some embodiments, the space in each bin is unlimited and thenumber of tags for a particular clip is also unlimited.

In some embodiments, a new clip extracted from a tagged clip may becreated with a tag if one of the extracted frames is tagged. FIG. 57illustrates such a clip creation process in three stages 5710-5730 ofthe GUI 5400. At the first stage 5710, the free-form display area 5405includes a media clip 5705 that is tagged with the tags “golf” and“vacation”. The term “golf” tags an early frame in the media clip whilethe term “vacation” tags a later frame in the media clip.

The second stage 5720 illustrates that a user has created a region ofinterest in the second half of media clip 5705 and extracted the regionof interest in order to create a new media clip. The region of interestincludes the frame that is tagged with the “vacation” tag. At stage5730, the new clip 5715 is created. As this new clip includes the frametagged with “vacation”, the clip is also tagged with the tag “vacation”.Accordingly, the bin 5725 for the “vacation” tag now includes two mediaclips (clips 5705 and 5715).

Some embodiments use the tags and bins for filtering of the free-formdisplay area. A user may select one of the bins, and the media-editingapplication will then only display clips or sequences in the selectedbin within the free-form display area. FIG. 58 conceptually illustratesa process 5800 of some embodiments for filtering the display area insuch a manner. The process 5800 will be described by reference to FIG.59, which illustrates the GUI 5400 in four stages 5910-5940. At thefirst stage 5910, the GUI 5400 is displaying all media clips in theproject within the free-form display area 5405.

As shown, the process 5800 begins by receiving (at 5805) a selection ofa tag. In some embodiments, a tag is selected when a user selects thebin for that tag. The user might click on the bin, tap the bin on atouch-screen display, navigate to the bin using a hotkey or series ofkeystrokes, etc. In some embodiments, a tag can also be selected byclicking on (or tapping) an instance of the tag in the free-form displayarea.

The process then identifies (at 5810) any media clips with at least oneframe tagged with the selected tag. In some embodiments, this will beeach media clip in the bin for the selected tag. The process thendisplays only the identified media clips in the free-form display area.Stage 5920 of FIG. 59 illustrates that a user has selected the “bike”bin, and as a result only the one media clip 5425 that is tagged withthe “bike” tag is displayed. At stage 5930, the user has selected the“car” bin, and thus only media clip 5625 is displayed. At stage 5940,the user has selected the “ride” tag, such that both clips 5425 and 5625are displayed. This feature may be used to identify similar media clipsand so that these clips may be more easily edited into a sequence. Insome embodiments, clicking in the browser off of the bins will restoreall of the media clips to the free-form display area. Other userinterface interactions, such as using a hotkey, will restore the mediaclips in various embodiments.

The embodiments described above by reference to FIGS. 54-59 use words astags. However, one of ordinary skill will recognize that someembodiments provide different types of tags in addition to words. Forinstance, in some embodiment, any alphanumeric characters, string, orany symbols can be used as tags.

As mentioned above, some embodiments create data structures for eachmedia clip in the free-form display area. An example of such a datastructure is described above by reference to FIG. 8, which shows a mediaclip data structure having a reference to a source media file and an inand out-point in that source file. As described above, however, someembodiments may insert additional information the data structures toaccount for tags, regions of interest, etc. FIG. 60 illustrates anexample data structure 6000 stored for a media clip in the free-formdisplay area. As shown, the media clip data 6000 includes a clip ID6005, clip name 6010, a reference 6015 to a source file, an in-point6020 and out-point 6025 in the source file, a set of tag information6030, a set of region of interest information 6035, and a poster frameidentification 6040.

The clip ID 6005 is a unique identification of the media clip. In someembodiments, a clip ID is unique within each of functional spaces (e.g.,organizing spaces and compositing spaces described above by reference toFIGS. 10-13) of a media-editing application. In other embodiments, it isunique within a media-editing application.

The clip name 6005 is a secondary identification of a media clip. Insome embodiments, it is the file name of a media source file (e.g.,“movie1.mov”, “movie2.avi”, etc.). When a media clip is duplicated in adisplay area in some embodiments, the duplicate copies of the two mediaclips have the same clip name but have different clip ID.

The reference to source file 6015 points to a location (e.g., on a localor network storage) at which the media source file (e.g., an audio file,video file, etc.) is stored. An example of reference to source file is amemory address. The in-point and out-point data elements 6020 and 6025store in and out-points within the timecode of the source media filethat respectively serves as the beginning and the of the media contentrepresented by the media clip. In some embodiments, the in andout-points are the beginning and the end of the whole duration of themedia clip by default. However, these in and out-points may be edited,as described below by reference to FIGS. 61-66. Thus, two media clipsmay point to a source file, but have different content due to differentin and out-points in the source file.

The tag information 6030 stores information about any tags that havebeen used to tag the media clip. The tag information includes the nameof each tag and, for each tag, a particular frame or set of frames ofthe media clip that are associated with the tag. The frame informationmay be stored as timecode information that refers to the timecode of thesource media file in some embodiments.

The region of interest information 6035 stores information about anyregions of interest that have been defined within the media clip, asdescribed above by reference to FIGS. 48-52 and 73. Each time a newregion is marked, some embodiments add data for this region to the datastructure 6000. For each region of interest, beginning and ending pointsare stored. This information may be stored as timecode information thatrefers to the timecode of the source media file in some embodiments.

The poster frame data element 6040 stores the location of the posterframe (i.e., representative thumbnail that is displayed in the free-formdisplay area to represent the media clip) when the media clip is a videoclip. In some embodiments, the poster frame of a media clip is the firstframe of the media clip by default, unless modified by a user. Themodification of the poster frame is described above by reference to FIG.20.

Although a media clip in some embodiments have many attributes asdescribed above, a media clip in other embodiments do not necessarilyhave all these attributes. For instance, a media clip in someembodiments does not necessarily have in and out-points or reference toa source file. In some such embodiments, the data structure for a mediaclip includes media content data.

v. Trimming Media Clips in Display Area

The media-editing application of some embodiments provides the abilityto modify the media clips in the free-form display area. For instance,some embodiments provide functionality that enables the user to trim themedia clip using the representation of the media clip in the free-formdisplay area. The user can remove portions of the source file from themedia clip by modifying the in and out-points of the media clip. One ofordinary skill in the art will note that such modifications to the mediaclips do not affect the actual source media files.

FIG. 61 illustrates an example of setting a new in-point for a mediaclip in a free-form display area of some embodiments. Specifically, FIG.61 illustrates setting a new in-point for a media clip 6100 by trimmingout the media content from the beginning of the media clip to a newin-point. For simplicity, the display area in which the media clip isdisplayed and edited is not illustrated in this figure. FIG. 61illustrates media clip 6100 at four different stages 6110-6140. FIG. 61also illustrates a time line 6150 of the media clip at each of thestages. The time line 6150 is not necessarily a part of the GUI thatincludes the display area, and is illustrated in this figure to show thedetails of the trimming operation.

In some embodiments, the media-editing application allows the user toset a new in-point for a media clip in the display area by selecting theleft edge of the thumbnail representing the media clip and moving theselected edge to a new location. The in-point will be moved by apercentage of the duration of the media clip that is based on (e.g.,equal to) the percentage of the width of the representative thumbnail.Thus, if a clip is one minute long and the left edge is moved halfwaythrough the thumbnail, the new in-point will be thirty seconds after theprevious in-point.

In the first stage 6110, media clip 6100 is displayed in the displayarea (not shown). In this example, media clip 6100 is a video clip witha five-minute duration as indicated by the time line 6150 illustrated onthe left side of the media clip 6100 in this figure. In this example,the representative frame used for the thumbnail representation is thefirst frame of the media clip.

In the second stage 6120, the user selects media clip 6100 for editing.In some embodiments, the user moves the cursor 6155 and performs aleft-click operation on the media clip. The border of the media clip isdisplayed in bold to indicate that the media clip is editable. The sideedges can be now selected and dragged inwards in order to trim the mediaclip.

The third stage 6130 illustrates that the user has dragged the left edgeof the clip to the right (inward) as indicated by the dotted hollowarrow by selecting the left edge with cursor 6155 and moving the cursor.This can also be performed on a touch-screen device by placing a fingeron the edge of the selected clip and moving it to the right. The userhas moved the left edge one fifth of the way through the thumbnailrepresentation, representing the one minute mark of the clip, asindicated by the time line 6150. In some embodiments, as the bold edgeis moved inwards, the original thin borders stay in place and retain thedimension of the clip representation. In addition, the frame displayedin the clip representation changes as though the left edge is aplayhead, so that the user can see the exact frame he wants to use asthe new in-point. The user releases the bold edge at this location toset the one minute mark as the new in-point of the media clip 6100.

The fourth stage 6140 illustrates the resulting media clip 6101 afterthe new in-point of the media clip 6100 is set at the point representingthe end of the first minute in the duration of the media clip. As aresult, the media clip 6100 now has a duration of four minutes ratherthan five minutes, as indicated by the timeline 6150. As all media clipsregardless of their actual durations are represented inuniformly-dimensioned rectangles in some embodiments, the length of themedia clip 6100 that used to represent five minutes now (i.e., the mediaclip 6101) represents four minutes. In addition, a new representativethumbnail is used at stage 6140, as the previous representative frame isno longer part of the media clip 6101. The default thumbnail is theframe at the new in-point of the media clip. In the case in which therepresentative frame is still part of the media clip after the trimmingoperation, some embodiments do not modify the representative frame afterthe edit.

In the case in which the media clip does not represent the entire sourcemedia file, some embodiments allow the user to make an in-point earlierby dragging the left edge to the left, outside of the thumbnail. Forinstance, the width of the thumbnail for media clip 6100 represents fourminutes after the edit operation illustrated in FIG. 61. If a user wereto later drag the left edge of the clip representation to the left adistance equal to one fourth of the width of the thumbnail, the in-pointwould be moved back to one minute earlier in the source file (i.e., themedia clip would then be the same as at stage 6110.

FIG. 62 illustrates an example of setting a new out-point for a mediaclip in a free-form display area of some embodiments. Specifically, FIG.61 illustrates setting a new out-point for a media clip 6100 by trimmingout the media content from the end of the media clip to a new out-point.For simplicity, the display area in which the media clip is displayedand edited is not illustrated in this figure. FIG. 62 illustrates mediaclip 6100 at four different stages 6210-6240. FIG. 62 also illustrates atime line 6150 of the media clip at each of the stages. The time line6150 is not necessarily a part of the GUI that includes the displayarea, and is illustrated in this figure to show the details of thetrimming operation.

In some embodiments, the media-editing application allows the user toset a new out-point for a media clip in the display area by selectingthe right edge of the thumbnail representing the media clip and movingthe selected edge to a new location. The out-point will be moved by apercentage of the duration of the media clip that is based on (e.g.,equal to) the percentage of the width of the representative thumbnail.Thus, if a clip is one minute long and the right edge is moved halfwaythrough the thumbnail, the new out-point will be thirty seconds beforethe previous out-point.

In the first stage 6210, media clip 6100 is displayed in the displayarea (not shown). In this example, media clip 6100 is a video clip witha five-minute duration as indicated by the time line 6150 illustrated onthe left side of the media clip 6100 in this figure. In this example,the representative frame used for the thumbnail representation is thefirst frame of the media clip.

In the second stage 6220, the user selects media clip 6100 for editing.In some embodiments, the user moves the cursor 6255 and performs aleft-click operation on the media clip. The border of the media clip isdisplayed in bold to indicate that the media clip is editable. The sideedges can be now selected and dragged inwards in order to trim the mediaclip.

The third stage 6230 illustrates that the user has dragged the rightedge of the clip to the left (inward) as indicated by the dotted hollowarrow by selecting the right edge with cursor 6255 and moving thecursor. This can also be performed on a touch-screen device by placing afinger on the edge of the selected clip and moving it to the left. Theuser has moved the right edge one-fifth of the way through the thumbnailrepresentation, representing the four minute mark (one minute from theend) of the clip, as indicated by the time line 6150. In someembodiments, as the bold edge is moved inwards, the original thinborders stay in place and retain the dimension of the cliprepresentation. In addition, the frame displayed in the cliprepresentation changes as though the right edge is a playhead, so thatthe user can see the exact frame he wants to use as the new out-point.The user releases the bold edge at this location to set the four minutemark as the new out-point of the media clip 6100.

The fourth stage 6240 illustrates the resulting media clip 6200 afterthe new out-point of the media clip 6100 is set at the pointrepresenting the end of the fourth minute in the duration of the mediaclip. As a result, the media clip 6200 now has a duration of fourminutes rather than five minutes, as indicated by the timeline 6150. Asall media clips regardless of their actual durations are represented inuniformly-dimensioned rectangles in some embodiments, the length of themedia clip 6100 that used to represent five minutes now (i.e., the mediaclip 6200) represents four minutes. As the previous representative frameis still part of the media clip 6100, this representative frame is stillused. In the case in which the representative frame is no longer part ofthe media clip after the trimming operation, some embodiments default tothe frame at the in-point of the media clip.

In the case in which the media clip does not represent the entire sourcemedia file, some embodiments allow the user to make an out-point laterby dragging the right edge to the right, outside of the thumbnail. Forinstance, the width of the thumbnail for media clip 6100 represents fourminutes after the edit operation illustrated in FIG. 62. If a user wereto later drag the right edge of the clip representation to the right adistance equal to one fourth of the width of the thumbnail, theout-point would be moved back to one minute later in the source file(i.e., the media clip would then be the same as at stage 6110).

vi. Editing Commands During Playback

FIGS. 61 and 62 described above illustrate examples of setting editpoints (i.e., in and out-points) of a media clip by pulling in the sideedges of the media clip in some embodiments. In those examples, the editpoints were set when the media clips were not playing back. In someembodiments, the user can set edit points of a media clip by invokingediting commands (e.g., by hitting or pressing hotkeys) while the mediaclip is being played back or skimmed. FIGS. 63-66, below, illustrateexamples of setting edit points of a media clip by invoking editingcommands while the media clip is being played back. In some embodiments,the operations to set edit points that are described below are performedwhen the media clip is being skimmed.

FIG. 63 illustrates setting an in-point of a media clip 6100 by invokinga command while the media clip 6100 is played back. FIG. 63 illustratesthe media clip 6100 at three different stages 6330-6350. FIG. 63 alsoillustrates a time line 6150 at each of the stages, which is notnecessarily a part of the GUI that includes the display area in whichthe media clip is displayed, and is illustrated in this figure to showthe details of the trimming operation.

In some embodiments, the media-editing application allows the user toset a new in-point for a media clip in the display area as the mediaclip is played back. During playback, a playhead moves along the mediaclip. When the playhead is at a location in the media clip at which theuser wants to set a new in-point for the media clip, the user can invokea command that will cause the frame at that location to be set as thenew in-point for the media clip. Thus, if a clip is one minute long andthe playhead is at the thirty second mark, the clip will be thirtyseconds long after the operation to move the in-point to the thirtysecond mark.

In the first stage 6330, the media clip 6100 is displayed in the displayarea (not shown). The media clip 6100 has a duration of five minutes, asindicated by the timeline 6150. A playhead 6360 indicates the positionof the currently displayed frame in the duration of the media clip thatis currently being played back. As mentioned above, the displayedlengths of different media clips do not necessarily represent the samelength of time in some embodiments, as each media clip is displayed asthe same size. In such cases, the playhead 6360 moves slower when it ison a longer media clip than when it is on a shorter media clip. In thisexample, the representative frames used for the thumbnailrepresentations for the media clip is the first frame of the media clip.At the first stage 6330, the user has just invoked a playback command tostart the playback process.

In the second stage 6340, the playback has reached the one minute markof the media clip 6100. At this point, the user invokes a command thatcauses the in-point of the media clip 6100 to be set to the currentlocation of the playhead 6360. In some embodiments, this edit operationis invoked through a hotkey (e.g., the “a” key).

The third stage 6350 illustrates that the media-editing application hasmodified the in-point of media clip 6100 such that the new in-point isthe timecode in the source media file for the clip that corresponds tothe one minute mark of the previous version of the clip before the edit.As a result, the resulting media clip 6101 now has a duration of fourminutes rather than five minutes, as indicated by the timeline 6150. Asall media clips regardless of their actual durations are represented inuniformly-dimensioned rectangles in some embodiments, the length of themedia clip 6100 that used to represent five minutes now (i.e., the mediaclip 6101) represents four minutes, as indicated by the timeline 6150and the dashed arrows pointing from the second stage 6340 to the thirdstage 6350. As shown in this stage, the playback of the media clip 6100continues as indicated by the dotted arrow point from the playhead 6360to the right edge of the media clip 6100. In this example, the endresult is the same as that of setting a new in-point by dragging theleft edge of the media clip 6100 in as described above by reference toFIG. 61.

Various operations may be performed in some embodiments in order to setout-points for single clips in a sequence and for a sequence as a whole.FIG. 64 illustrates setting an in-point of a media clip 6100 by invokinga command while the media clip 6100 is played back. FIG. 64 illustratesthe media clip 6100 at three different stages 6430-6450. FIG. 64 alsoillustrates a time line 6150 at each of the stages, which is notnecessarily a part of the GUI that includes the display area in whichthe media clip is displayed, and is illustrated in this figure to showthe details of the trimming operation.

In some embodiments, the media-editing application allows the user toset a new out-point for a media clip as the media clip is played back.During playback, a playhead moves along the media clip. When theplayhead is at a location in a media clip at which the user wants to seta new out-point for the media clip, the user can invoke a command thatwill cause the frame at that location to be set as the new out-point forthe media clip. Thus, if a clip is one minute long and the playhead isat the thirty second mark, the clip will be thirty seconds long afterthe operation to move the out-point to the thirty second mark.

In the first stage 6430, the media clip 6100 is displayed in the displayarea (not shown). The media clip 6100 has a duration of five minutes, asindicated by the timeline 6150. A playhead 6460 indicates the positionof the currently displayed frame in the duration of the media clip thatis currently being played back. As mentioned above, the displayedlengths of media clips do not necessarily represent the same length oftime in some embodiments, as each media clip is displayed as the samesize. In such cases, the playhead 6460 moves slower when it is on alonger media clip than when it is on a shorter media clip of thesequence. In this example, the representative frames used for thethumbnail representations for the media clip is the first frame of themedia clip. At the first stage 6430, the user has just invoked aplayback command to start the playback process.

In the second stage 6440, the playback has reached the four minute markof the media clip 6100. At this point, the user invokes a command thatcauses the out-point of the media clip 6405 to be set to the currentlocation of the playhead. In some embodiments, this edit operation isinvoked through a hotkey (e.g., the “s” key).

The third stage 6450 illustrates that the media-editing application hasmodified the out-point of clip 6100 such that the new out-point is thetimecode in the source media file for that clip that corresponds to thefour minute mark of the clip. As a result, the resulting media clip 6200now has a duration of four minute rather than five minutes, as indicatedby the timeline 6150. As all media clips regardless of their actualdurations are represented in uniformly-dimensioned rectangles in someembodiments, the length of the media clip 6100 that used to representfive minutes now (i.e., the media clip 6200) represents four minutes, asindicated by the timeline 6150 and the dashed arrows pointing from thesecond stage 6440 to the third stage 6450. As the previousrepresentative frame is still part of the media clip 6100, thisrepresentative frame is still used. In this example, the end result isthe same as that of setting a new out-point by dragging the right edgeof the media clip 6100 in as described above by reference to FIG. 62.

FIG. 65 illustrates extending a media clip 6500 by extending itsout-point by invoking a command while the media clip 6500 is playedback. FIG. 65 illustrates the media clip 6500 at four different stages6520-6550. FIG. 65 also illustrates a time line 6555, marker 6570, andarrow 6565 at each of the stages, which are not necessarily a part ofthe GUI that includes the display area in which the sequence isdisplayed, and are illustrated in this figure to show the details of thetrimming operation.

In some embodiments, when a media clip does not contain the entirety ofits source media file, the media-editing application allows the user toset a new out-point for the media clip as the media clip is played back.During playback, a playhead moves along the media clip. Before theplayhead reaches the end of the media clip, the user can invoke acommand that will cause the media clip to continue playing content fromits source file after the current out-point is reached. When theplayhead reaches a location in the media clip source at which the userwants to set a new out-point for the media clip, the user can invokes acommand that will cause the frame at that location to be set as the newout-point for the media clip.

In the first stage 6520, the media clip 6500 is displayed in the displayarea (not shown). The media clip 6500 has a duration of three minutes,as indicated by the timeline 6555. As shown, however, the source mediafile for clip 6500 has a duration of five minutes, and the currentout-point of the media clip 6500 is set to the timecode that correspondsto the three minute mark of the source media file. The arrow 6565 pointsto the current location of the out-point of the media clip 6500 alongthe duration of the media source file, while marker 6570 indicates thelocation of the playhead in the media source file. At the first stage6520, the user has just invoked the playback command to start theplayback process.

In the second stage 6530, the playback has reached the two minute markof the media clip 6500. At this point, the user invokes a command thatinstructs the media-editing application to continue playing past thecurrent out-point of the currently playing clip 6500. In someembodiments, this operation is invoked through a hotkey (e.g., the “d”key).

In the third stage 6540, the playback has reached the three minute markthat marks the end of the media clip 6500 according to its setout-point. The playhead is now at the end of the thumbnail representingthe media clip 6500, and stays at that point as the media source filecontinues to play in the thumbnail representation. However, because thecommand was invoked at stage 6530, the media-editing applicationcontinues to play media data from the source file. At this point, theplayhead marker 6570 is at the three minute mark, as is the out-pointarrow 6565. In some embodiments, the playback of the media clip 6500continues until the user releases the hotkey or the media source fileends. If the hotkey is released before the current out-point, thenplayback continues as it would have if the command had not been invokedat all.

In the fourth stage 6550, the playback is now at minute four of thesource media file, as indicated by playhead marker 6570. As this is pastthe previous out-point, the out-point of the media file 6500 has beenextended to this four minute mark, as indicated by out-point arrow 6565.The resulting media clip is media clip 6501. The out-point will beextended up until the user releases the hotkey in some embodiments orthe source file ends, in which case the out-point will be set to thetimecode marking the end of the source file.

In some embodiments, operations may be performed to split a media clipin a sequence into two clips. FIG. 66 illustrates splitting a media clip6605 into two media clips 6615 and 6620 by invoking a command while themedia clip 6605 is played back. FIG. 66 illustrates the media clip 6605at three different stages 6630-6640. FIG. 66 also illustrates a timeline 6645 at each of the stages, which is not necessarily a part of theGUI that includes the display area in which the sequence is displayed,and is illustrated in this figure to show the details of the editingoperation

In some embodiments, the media-editing application allows the user tosplit a media clip that is part of a sequence in the display area as thesequence is played back. During playback, a playhead moves along themedia clips in the sequence. When the playhead is at a location in amedia clip at which the user wants to split the media clip, the user caninvoke a command that will cause the frame at that location to be set asthe new out-point for the playing media clip and as the in-point for anew media clip that runs from that in-point to the previous out-pointfor the original media clip.

In the first stage 6630, the media clip 6605 is displayed in the displayarea (not shown). The media clip 6605 has a duration of five minutes, asindicated by the timeline 6645. At the first stage 6630, the user hasjust invoked a playback command to start the playback process. In thesecond stage 6635, the playback has reached the two minute mark of themedia clip 6605. At this point, the user invokes a command that causesthe currently playing clip 6605 to be split into two clips. In someembodiments, this edit operation is invoked through a hotkey (e.g., the“b” key).

The third stage 6640 illustrates that the media-editing application hassplit the media clip 6605 into clips 6615 and 6620. The clip 6615 willhave the same in-point as media clip 6605 and an out-point two minuteslater (at the frame playing when the split clip command was invoked).The clip 6620 will have an in-point one frame after the out-point ofclip 6615 and the same out-point as media clip 6605. Some embodimentsuse the frame playing when the split clip command is invoked as thein-point of the second clip and the frame prior to that as the out-pointof the first clip. The representative frame for clip 6615 is the same asthat of clip 6605, while the representative frame for clip 6620 is thatat its in-point. As all media clips regardless of their actual durationsare represented in uniformly-dimensioned rectangles in some embodiments,the length of the two media clips that represent the same actual sourcecontent as the single clip 6605 has doubled in size.

As described above, some embodiments perform editing operations on amedia clip based on editing commands that the user invokes. FIG. 67illustrates an example of a conceptual process 6700 of some embodimentsfor performing a set of editing operations on a media clip in afree-form display area. In some embodiments, the process 6700 isperformed by an media-editing application that provides a free-formdisplay area that displays the media clip. In some such embodiments, theprocess 6700 is performed when the user has selected a media clip toplay and has invoked a command to start playback of the media clip. Insome cases, the selected media clip is part of a sequence. In othercases, the selected media clip is an individual media clip.

The process 6700 begins by starting (at 6710) playback of the media clipin response to the user invoking a command to start playback of a mediaclip in the display area. The media clip is played back from thebeginning of the media clip (e.g., the beginning of the media sourcefile or from a set in-point) or from a point within the media clip fromwhich the user has chosen to play back. In some embodiments, the userchooses the point to play back from by placing a playhead on a locationalong the length of the media clip that represent the point and invokinga forward playback command. In some cases, the user plays back the mediaclip backwards by invoking a backward playback command. Some examples ofinvoking forward and backward playback commands are described above byreference to FIG. 2. In those cases when the media clip is part of asequence, the media clip is played back after a neighboring media clipin the sequence is played back (i.e., a playhead moves to the media clipfrom the neighboring media clip) unless the user chooses to play backthe media clip.

The process 6700 then advances (at 6715) to the next frame in the mediaclip to display in the display area. The next frame is a frame thatcomes chronologically after the current frame of the media clip beingdisplayed when the media clip is being played back forwards. When themedia clip is being played backwards, the next frame is the frame thatcomes chronologically before the current frame of the media clip beingdisplayed. The process 6700 displays the next frame and the framebecomes the current frame being displayed.

Next, the process 6700 determines (at 6720) whether a command that cutsmedia content (e.g., image frames) of the media clip from the currentin-point to the current frame is received. When the process 6700determines that such a command is received, the process 6700 sets (at6725) the in-point of the media clip to the current frame. As describedabove, the in-point of a media clip in some embodiments is the firstframe of the media clip unless it is reset to another point in the mediaclip. In some embodiments, the process 6700 sets an in-point by changingthe in-point data element of a data structure, such as the one describedabove by reference to FIG. 8. As described above, the next time themedia clip is played back, the image frames cut out will not be playedback and playback will start from the set in-point. A detailed exampleof an operation to set a new in-point is described above by reference toFIG. 63. The process 6700 then proceeds to 6780 which will be describedfurther below.

When the process 6700 determines (at 6720) that it has not received acommand to set a new in-point of the media clip, the process determines(at 6730) whether a command that cuts media content of the media clipfrom the current frame to the current out-point is received. When theprocess 6700 determines that such a command is received, the process6700 sets (at 6733) the out-point of the media clip to the currentframe. As describe above, the out-point of a media clip in someembodiments is the last frame of the media clip unless it is reset toanother point in the media clip. In some embodiments, the process 6700sets an out-point by changing the in-point data element of a datastructure, such as the one described above by reference to FIG. 8. Asdescribed above, next time the media clip is played back, the imageframes cut out will not be played back. A detailed example of anoperation to set a new out-point is described above by reference to FIG.64. The process 6700 then proceeds to 6780 which will be describedfurther below.

When the process 6700 determines (at 6730) that it has not received acommand to set a new out-point of the media clip, the process 6700determines (at 6735) whether a command to split the media clip into twoseparate media clips at the current frame is received. When the process6700 determines that such a command is received, the process 6700 splitsthe media clip into two different media clips. In some embodiments, theprocess creates two new media clips. One of the new media clips willrepresent the media content of the media clip being split from thebeginning of the media clip to the current frame of the media clip. Theother new media clip will represent the rest of the media content in themedia clip. A detailed example of an operation to split a media clip isdescribed above by reference to FIG. 66. The process 6700 then proceedsto 6780 which will be described further below.

When the process 6700 determines (at 6735) that it has not received acommand to split the media clip, the process 6700 determines (at 6745)whether a command to extend an approaching edit point of the media clipis received. An approaching edit point is the edit point (e.g., in-pointand out-point) that a playhead is getting closer to while the media clipis being played back forwards or backwards. That is, when the media clipis being played backwards the in-point of the media clip is theapproaching edit point. When the media clip is being played forward, theout-point of the media clip is the approaching edit point. When theprocess 6700 determines that such a command is not received, the process6700 then proceeds to 6780 which will be described further below.

When the process 6700 determines that such a command is received, theprocess 6700 determines (at 6750) whether the command is released (e.g.,whether the user is holding down a hotkey that invokes the command).When the process 6700 determines that the user is still holding down thehotkey, the process proceeds to 6755 which will be described furtherbelow.

When the process 6700 determines that the user is no longer holding downthe hotkey and thereby releasing the command, the process 6700determines (at 6760) whether it has run out of media content to play.The process 6700 runs out of media content to play when it has playedback forwards and has reached the last frame of the source media file.The process 6700 also runs out of media content to play when it hasplayed back backwards and has reached the first frame of the sourcemedia file.

When the process 6700 determines that it has not run out of mediacontent to play back, the process 6700 advances (at 6765) to the nextframe in the playback direction and then goes back to 6750 to determinewhether the user is still holding down the hotkey. When the process 6700determines that it has run out of media content to play back, theprocess 6700 sets (at 6770) the current frame (e.g., the first or thelast frame of the media clip) as a new edit point (i.e., the edit pointhas been extended to the new edit point.) and then the process 6700ends.

When the process 6700 determines (at 6750) that the user is stillholding down the hotkey, the process 6700 determines (at 6755) whetherthe current frame is past the edit point that was approaching at thetime the command was invoked (i.e., at the time the user started topress and hold down the hotkey). If the process 6700 determines that thecurrent frame is not past the edit point, the process 6700 sets (at6775) the current frame as new edit point. Otherwise, the processproceeds to 6780.

At 6780, the process 6700 then determines whether the playback hasstopped. The process 6700 stops when it runs out of media content toplay back. The process 6700 also stops when it receives a command tostop the playback of the media clip (e.g., user's invoking a stopcommand). When the process 6700 determines that the playback has notstopped, the process 6700 return to 6715 to continue advancing to thenext frame.

In this example, a media file that the process 6700 performs editoperations on represents media content in the form of image frames.However, one of ordinary skill will recognize that this conceptualprocess is applicable to media files that represent other type of mediacontent such as audio data.

FIGS. 63-66 illustrate examples of editing a media clip in a sequenceaccording to some embodiments by invoking editing commands while themedia clip is being played back. In some embodiments, these commands canbe invoked while the sequence is played back in full screen mode, whichis described above by reference to FIG. 44.

Moreover, some embodiments allow the editing commands to be invokedwhile the sequence is displayed in a free-form display area (e.g., thefree-form display area 700 described above by reference to FIG. 7) thatis the only compositing area or one of multiple compositing areas of amedia-editing application. Also, some embodiments allow editing a mediaclip by invoking editing commands while the media clip is displayed in acomposite display area (such as the composite display area 1120described above by reference to FIG. 11) that is the only compositingarea or one of multiple compositing areas of a media-editingapplication.

B. Editing Composite Sequences

The above subsection described various editing operations that can beperformed on media clips in some embodiments (e.g., trimming, splitting,extending, etc.). Some embodiments also perform a variety of similarediting operations on composite sequences, the creation of which wasdescribed above in Section III.

i. Setting In-Points

Various operations may be performed in some embodiments in order to setin-points for single clips in a sequence and for a sequence as a whole.FIG. 68 illustrates setting an in-point of a media clip 6805 by invokinga command while a sequence 6820 of which the media clip 6805 is a partis played back. FIG. 68 illustrates sequence 6820 at three differentstages 6830-6850. FIG. 68 also illustrates a time line 6855 at each ofthe stages, which is not necessarily a part of the GUI that includes thedisplay area in which the sequence is displayed, and is illustrated inthis figure to show the details of the trimming operation.

In some embodiments, the media-editing application allows the user toset a new in-point for a media clip that is part of a sequence in thedisplay area as the sequence is played back. During playback, a playheadmoves along the media clips in the sequence. When the playhead is at alocation in a media clip at which the user wants to set a new in-pointfor the media clip, the user can invoke a command that will cause theframe at that location to be set as the new in-point for the media clip.Thus, if a clip is one minute long and the playhead is at the thirtysecond mark, the clip will be thirty seconds long after the operation tomove the in-point to the thirty second mark.

In the first stage 6830, sequence 6820 is displayed in the display area(not shown). In this example, sequence 6820 is a sequence of media clips6800, 6805, and 6810, arranged in that order. The media clip 6800 has aduration of five minutes, media clip 6805 has a duration of fourminutes, and media clip 6810 has a duration of eight minutes, asindicated by the timeline 6855 illustrated along the sequence.

A playhead 6860 indicates the position of the currently displayed framein the duration of the media clip that is currently being played back.As mentioned above, each displayed length of media clip in a sequencedoes not necessarily represent the same length of time in someembodiments, as each media clip is displayed as the same size. In suchcases, the playhead 6860 moves slower when it is on a longer media clipthan when it is on a shorter media clip of the sequence. In thisexample, the representative frames used for the thumbnailrepresentations for each of the media clips are the first frame of themedia clips. At the first stage 6830, the user has just invoked aplayback command to start the playback process.

In the second stage 6840, the playback has reached the three minute markof the second media clip 6805 in the sequence 6820. At this point, theuser invokes a command that causes the in-point of the currently playingclip 6805 to be set to the current location of the playhead. In someembodiments, this edit operation is invoked through a hotkey (e.g., the“a” key). This hotkey is the same as that used to set the in-point of asingle media clip in some embodiments.

The third stage 6850 illustrates that the media-editing application hasmodified the in-point of clip 6805 such that the new in-point is thetimecode in the source media file for that clip that corresponds to thethree minute mark of the previous version of the clip before the edit.As a result, the resulting media clip 6806 now has a duration of oneminute rather than four minutes, as indicated by the timeline 6855. Asall media clips regardless of their actual durations are represented inuniformly-dimensioned rectangles in some embodiments, the length ofmedia clip 6805 that used to represent four minutes now (i.e., the mediaclip 6806) represents one minute. In addition, a new representativethumbnail is used at stage 6850, as the previous representative frame isno longer part of the media clip 6805. The default thumbnail is theframe at the new in-point of the media clip. In the case in which therepresentative frame is still part of the media clip after the trimmingoperation, some embodiments do not modify the representative frame afterthe edit.

FIG. 69 illustrates cutting a sequence 6820 so as to remove any mediafrom the beginning of the sequence to a frame that is currently beingplayed back. FIG. 69 illustrates sequence 6820 at three different stages6930-6950. FIG. 69 also illustrates a time line 6855 at each of thestages, which is not necessarily a part of the GUI that includes thedisplay area in which the sequence is displayed, and is illustrated inthis figure to show the details of the trimming operation.

In some embodiments, the media-editing application allows the user tocut out the beginning of a sequence up to a particular point as thesequence is played back. Doing so will modify the in-point of the mediaclip that is playing back when the command is invoked, and will removeany previous clips from the sequence. During playback, a playhead movesalong the media clips in the sequence. When the playhead is at alocation in a media clip at which the user wants the sequence to start,the user can invoke a command that will cause the frame at that locationto be set as the new in-point for the media clip, and will cause theprevious clips to be removed from the sequence.

In the first stage 6930, sequence 6820 is displayed in the display area(not shown). In this example, sequence 6820 is a sequence of media clips6800, 6805, and 6810, arranged in that order. A playhead 6860 indicatesthe position of the currently displayed frame in the duration of themedia clip that is currently being played back. As mentioned above, eachdisplayed length of media clip in a sequence does not necessarilyrepresent the same length of time in some embodiments, as each mediaclip is displayed as the same size. In such cases, the playhead 6860moves slower when it is on a longer media clip than when it is on ashorter media clip of the sequence. In this example, the representativeframes used for the thumbnail representations for each of the mediaclips are the first frame of the media clips. At the first stage 6930,the user has just invoked a playback command to start the playbackprocess.

In the second stage 6940, the playback has reached the three minute markof the second media clip 6805 in the sequence 6820. At this point, theuser invokes a command that causes the in-point of the currently playingclip 6805 to be set to the current location of the playhead, and theclip 6800 to be cut from the sequence 6820. In some embodiments, thisedit operation is invoked through a hotkey (e.g., the “z” key). Thishotkey is the same as that used to set the in-point of a single mediaclip in some embodiments.

The third stage 6950 illustrates that the media-editing application hasremoved the clip 6800 and modified the in-point of clip 6805 such thatthe new in-point is the timecode in the source file for that clip thatcorresponds to the three minute mark of the previous version of the clipbefore the edit. As a result, the resulting media clip 6905 now has aduration of one minute rather than four minutes, as indicated by thetimeline 6855. In addition, a new representative thumbnail is used atstage 6950, as the previous representative frame is no longer part ofthe media clip 6805. The default thumbnail is the frame at the newin-point of the media clip. In the case in which the representativeframe is still part of the media clip after the trimming operation, someembodiments do not modify the representative frame after the edit. Thesequence now only consists of two clips and is nine minutes long. Insome embodiments, the media clip 6800 is deleted, while in otherembodiments it is merely split off from the sequence 6820 and remains inthe free-form display area.

ii. Setting Out-points

Various operations may be performed in some embodiments in order to setout-points for single clips in a sequence and for a sequence as a whole.FIG. 70 illustrates setting an in-point of a media clip 6800 by invokinga command while a sequence 6820 of which the media clip 6800 is a partis played back. FIG. 70 illustrates sequence 6820 at three differentstages 7030-7050. FIG. 70 also illustrates a time line 6855 at each ofthe stages, which is not necessarily a part of the GUI that includes thedisplay area in which the sequence is displayed, and is illustrated inthis figure to show the details of the trimming operation.

In some embodiments, the media-editing application allows the user toset a new out-point for a media clip that is part of a sequence in thedisplay area as the sequence is played back. During playback, a playheadmoves along the media clips in the sequence. When the playhead is at alocation in a media clip at which the user wants to set a new out-pointfor the media clip, the user can invoke a command that will cause theframe at that location to be set as the new out-point for the mediaclip. Thus, if a clip is one minute long and the playhead is at thethirty second mark, the clip will be thirty seconds long after theoperation to move the out-point to the thirty second mark.

In the first stage 7030, sequence 6820 is displayed in the display area(not shown).

A playhead 6860 indicates the position of the currently displayed framein the duration of the media clip that is currently being played back.As mentioned above, each displayed length of media clip in a sequencedoes not necessarily represent the same length of time in someembodiments, as each media clip is displayed as the same size. In suchcases, the playhead 6860 moves slower when it is on a longer media clipthan when it is on a shorter media clip of the sequence. In thisexample, the representative frames used for the thumbnailrepresentations for each of the media clips are the first frame of themedia clips. At the first stage 7030, the user has just invoked aplayback command to start the playback process.

In the second stage 7040, the playback has reached the four minute markof the first media clip 6800 in the sequence 6820. At this point, theuser invokes a command that causes the out-point of the currentlyplaying clip 6805 to be set to the current location of the playhead. Insome embodiments, this edit operation is invoked through a hotkey (e.g.,the “s” key). This hotkey is the same as that used to set the out-pointof a single media clip in some embodiments.

The third stage 7050 illustrates that the media-editing application hasmodified the out-point of clip 6800 such that the new out-point is thetimecode in the source media file for that clip that corresponds to thefour minute mark of the clip. As a result, the resulting media clip 7000now has a duration of four minute rather than five minutes, as indicatedby the timeline 6855. As all media clips regardless of their actualdurations are represented in uniformly-dimensioned rectangles in someembodiments, the length of media clip 6800 that used to represent fiveminutes now represents four minutes. As the previous representativeframe is still part of the media clip 6800, this representative frame isstill used. In the case in which the representative frame is no longerpart of the media clip after the trimming operation, some embodimentsdefault to the frame at the in-point of the media clip.

FIG. 71 illustrates extending a media clip 7100 by extending itsout-point by invoking a command while a sequence 7115 of which the mediaclip 7100 is a part is played back. FIG. 71 illustrates sequence 7115 atfour different stages 7120-7150. FIG. 71 also illustrates a time line7155, marker 7170, and arrow 7165 at each of the stages, which are notnecessarily a part of the GUI that includes the display area in whichthe sequence is displayed, and are illustrated in this figure to showthe details of the trimming operation.

In some embodiments, when a media clip in a sequence does not containthe entirety of its source media file, the media-editing applicationallows the user to set a new out-point for the media clip as thesequence is played back. During playback, a playhead moves along themedia clips in the sequence. Before the playhead reaches the end of themedia clip, the user can invoke a command that will cause the media clipto continue playing content from its source file after the currentout-point is reached. When the playhead reaches a location in the mediaclip source at which the user wants to set a new out-point for the mediaclip, the user can invoke a command that will cause the frame at thatlocation to be set as the new out-point for the media clip.

In the first stage 7120, sequence 7115 is displayed in the display area(not shown). In this example, sequence 7115 is a sequence of media clips7100, 7105, and 7110, arranged in that order. The media clip 7100 has aduration of three minutes, media clip 7105 has a duration of fourminutes, and media clip 7110 has a duration of eight minutes, asindicated by the timeline 7155 illustrated along the sequence. As shown,however, the source media file for clip 7100 has a duration of fiveminutes, and the current out-point of clip 7100 is set to the timecodethat corresponds to the three minute mark of the source media file. Thearrow 7165 points to the current location of the out-point of the mediaclip along the duration of the media source file, while marker 7170indicates the location of the playhead in the media source file. At thefirst stage 7120, the user has just invoked the playback command tostart the playback process.

In the second stage 7130, the playback has reached the two minute markof the first media clip 7100 in the sequence 7115. At this point, theuser invokes a command that instructs the media-editing application tocontinue playing past the current out-point of the currently playingclip 7100. In some embodiments, this operation is invoked through ahotkey (e.g., the “d” key). This hotkey is the same as that used toextend the out-point of a single media clip in some embodiments.

In the third stage 7140, the playback has reached the three minute markthat marks the end of the media clip 7100 according to its setout-point. The playhead 7160 is now at the end of the thumbnailrepresenting the media clip, and stays at that point as the media sourcefile continues to play in the thumbnail representation. However, becausethe command was invoked at stage 7130, the media-editing applicationcontinues to play media data from the source file. At this point, theplayhead marker 7170 is at the three minute mark, as is the out-pointarrow 7165. In some embodiments, the playback of clip 7100 continuesuntil the user releases the hotkey or the media source file ends. If thehotkey is released before the current out-point, then playback continuesas it would have if the command had not been invoked at all.

In the fourth stage, the playback is now at minute four of the sourcemedia file, as indicated by playhead marker 7170. As this is past theprevious out-point, the out-point of the media file 7100 has beenextended to this four minute mark, as indicated by out-point arrow 7165.The resulting media clip is media clip 7101 and the resulting sequenceis sequence 7116.

The out-point will be extended up until the user releases the hotkey insome embodiments or the source file ends, in which case the out-pointwill be set to the timecode marking the end of the source file.

Similarly, an in-point of a media clip (e.g., a video clip) in asequence (or a single media clip not part of a sequence) is extendablein some embodiments. That is, these embodiments allow the user to set anew in-point for the media clip to a point in a portion of the sourcemedia file that is not currently represented by the media clip while themedia clip is being played back. Some embodiments require the media clipto be played back backwards (i.e., reverse chronology) for the user toinvoke a command that instructs the media-editing application tocontinue playing backward past the current in-point of the playing clip.

In some such embodiments, the user invokes separate commands forstarting backward playback of the media clip and for extending thein-point. The user in some cases stops or pauses forward playback of theclip and then starts playing backwards. In other cases, the userreverses the playback direction to the backward direction withoutstopping the forward playback. Other embodiments provide a singlecommand that allows the user to playback the media clip backwards andextend the in-point. Such a command is invokable regardless of theplayback status of the media clip (e.g., stopped, paused, playingbackwards or forwards). Regardless of how the in-point extension commandis invoked, this command results in the redefining of the in-point toinclude in a media clip an earlier portion (e.g., earlier frames fromthe video content) of the source media file.

iii. Splitting a Clip

In some embodiments, operations may be performed to split a media clipin a sequence into two clips. FIG. 72 illustrates splitting a media clip7205 into two media clips 7215 and 7220 by invoking a command while asequence 7260 of which the media clip 7205 is a part is played back.FIG. 72 illustrates the sequence 7260 at three different stages7230-7240. FIG. 72 also illustrates a time line 7245 at each of thestages, which is not necessarily a part of the GUI that includes thedisplay area in which the sequence is displayed, and is illustrated inthis figure to show the details of the editing operation.

In some embodiments, the media-editing application allows the user tosplit a media clip that is part of a sequence in the display area as thesequence is played back. During playback, a playhead moves along themedia clips in the sequence. When the playhead is at a location in amedia clip at which the user wants to split the media clip, the user caninvoke a command that will cause the frame at that location to be set asthe new out-point for the playing media clip and as the in-point for anew media clip that runs from that in-point to the previous out-pointfor the original media clip.

In the first stage 7230, sequence 7260 is displayed in the display area(not shown). In this example, sequence 7260 is a sequence of media clips7200, 7205, and 7210, arranged in that order. The media clip 7200 has aduration of five minutes, media clip 7205 has a duration of six minutes,and media clip 7210 has a duration of eight minutes, as indicated by thetimeline 7245 illustrated along the sequence. At the first stage 7230,the user has just invoked a playback command to start the playbackprocess.

In the second stage 7235, the playback has reached the two minute markof the second media clip 7205 in the sequence 7260. At this point, theuser invokes a command that causes the currently playing clip 7205 to besplit into two clips. In some embodiments, this edit operation isinvoked through a hotkey (e.g., the “b” key). This hotkey is the same asthat used to split a single media clip in some embodiments.

The third stage 7240 illustrates that the media-editing application hassplit the media clip 7205 into clips 7215 and 7220. The clip 7215 willhave the same in-point as media clip 7205 and an out-point two minuteslater (at the frame playing when the split clip command was invoked).The clip 7220 will have an in-point one frame after the out-point ofclip 7215 and the same out-point as media clip 7205. Some embodimentsuse the frame playing when the split clip command is invoked as thein-point of the second clip and the frame prior to that as the out-pointof the first clip. The representative frame for clip 7215 is the same asthat of clip 7205, while the representative frame for clip 7220 is thatat its in-point. As all media clips regardless of their actual durationsare represented in uniformly-dimensioned rectangles in some embodiments,the length of the two media clips that represent the same actual sourcecontent as the single clip 7205 has doubled in size, and the sequence7260 now includes four clips rather than three.

FIGS. 68-72 illustrate examples of dynamically editing a sequence ofvideo clips in a free-form display area of some embodiments. That is,these examples show editing a sequence of video clips by invokingediting commands while the sequence is being played back within thethumbnails of the sequence in the free-form display area. In someembodiments, these editing commands can also be invoked while thesequence is played back in a bigger viewing area such as the viewingarea 4350 or in a full screen mode, an example of which is describedabove by reference to FIG. 44. The ability to edit a sequence of videoclips while playing back the sequence in the viewing area 4350 or in afull screen mode is beneficial to a user of the media-editingapplication because the user is able to see the effects or results ofthe edits immediately (or as the sequence is being played back) and theeffects viewed in the viewing area 4350 or in a full screen modeprovides different experience than those viewed in a small viewing area(e.g., thumbnails of the sequence) of the media editing application.

As an example of seeing the effects of edits in the viewing area 4350 orin a full screen mode immediately, while playing back the sequence ofvideo clips in the viewing area 4350 or in a full screen mode, someembodiments allow the user to trim the video clip currently being playedback to the end of the clip (i.e., setting a new out-point) by pressinga hotkey. Upon receiving the hotkey, the full screen playback quicklytransitions to the first frame in the next video clip in the sequence.As such, the user is able to see the effects of the edits as the user ismaking edits in the viewing area 4350 or in a full screen mode.

Extending an out-point of a media clip in a sequence while the sequenceis played back in the viewing area 4350 or in a full screen mode in someembodiments shows a similar transition to the next video clip from a newout-point. As soon as the user releases a hotkey when a desired frame isdisplayed during the playback of the media clip in some embodiments, theplayback transitions to the first frame of the next video clip in thesequence and the playback continues from that frame on.

iv. Marking a Region of Interest

Some embodiments enable a user to mark a region of interest of asequence in the free-form display area. As described above, a region ofinterest is a temporal portion of a sequence that is selected by theuser. FIG. 73 illustrates marking a region of interest of sequence 7320while the sequence is played back. FIG. 73 illustrates sequence 7320 atfour stages 7330-7355.

The first stage 7330 illustrates that the user has begun playback of thesequence 7320. The sequence 7320 consists of media clips 7300, 7305,7310, and 7315, arranged in that order. At the second stage 7340 theplayhead is approximately two-fifths of the way through the first clip7300 when the user invokes a command to start defining and marking aregion of interest. In some embodiments, the user holds down a hotkeyduring playback in order to define the region of interest.

At the third stage 7350, the user has not yet invoked a command to endthe region of interest (e.g., releasing the hotkey). At this point, theplayhead (and thus the end of the region of interest) is most of the waythrough the first media clip in the sequence 7320. At the fourth stage7355, the user invokes the command to end the region of interest, whichincludes portions of both clip 7300 and clip 7305 at this point. Thecommand is invoked by releasing the hotkey in some embodiments.

The region of interest is now marked on the sequence by the resultingmarker 7366, which spans from the start point in the first clip to theend point in the second clip. Owing to the difference in duration of thedifferent clips, equal distances along the marker at different portionsof the marker may represent different temporal lengths of media. In someembodiments, as described above for a single clip, the marker can beselected and dragged out of the sequence (or copied and pasted) in orderto create a new single clip (if spanning only one clip) or sequence (ifspanning multiple clips).

This Sub-section IV.B has so far described editing operations oncomposite sequences that are performed when the sequences are displayedand or played back in a free-form display area. Some embodiments allowthese operations to be performed on composite sequences when they aredisplayed in a composite display area such as 1120 described above byreference to FIG. 11. Several specific examples of the editingoperations that some embodiments allow to be performed on sequencesdisplayed in a composite display area are now described below byreference to the next two FIGS. 74 and 75.

FIG. 74 illustrates a GUI 7400 of a media-editing application of someembodiments that includes a viewing area 7405 and a composite displayarea 7410. Specifically, FIG. 74 illustrates an example of setting anin-point of a media clip 7420 by invoking a command while a sequence7430 of which the media clip 7420 is a part is played back. FIG. 74illustrates the GUI 7400 at three different stages 7435-7445.

The composite display area 7410 is similar to the composite display area1120 described above by reference to FIG. 11. The composite display areaincludes multiple tracks 7455-7465 that span a time line 7470 anddisplays a graphical representation of the composite presentation bydisplaying media clips 7415-7425 (or sequence 7430) that form thecomposite presentation. As described above, media clips in someembodiments are represented in a composite display area as a set ofrectangular representations where the length of the rectangle provides avisual indicator of the length of the associated clip.

The composite display area 7410 is the only compositing space in the GUI7400 of the media-editing application in some embodiments. Otherembodiments provide the composite display area 7410 as one of severalcomposite display areas of the GUI 7400. One such composite display areaof the GUI 7400 may be the free-form display area 1305 described abovein 1305.

The timeline 7470 shows where in the duration of the compositepresentation the media clips and sequences are located. Also, as thelength of a media clip or a sequence is proportional to its actualduration in the composite display area, a media clip or a sequence'sspan along the timeline indicates the begin and end of the duration ofthe media clip within the duration of the composite presentation.

A viewing area 7405 is an area in the GUI 7400 for displaying playbackof a media clip, a sequence of media clips, or a composite presentation.The viewing area 7405 is similar to ones described above by reference toFIGS. 10-13. In some embodiments, the viewing area 7405 is one ofseveral areas for displaying playback of media clips, a sequence, or acomposite presentation in the tracks. In some such embodiments, aviewing area is assigned to each track to display playback of the mediaclips or sequences in the track, in addition to a viewing area dedicatedto display playback of a composite presentation (i.e., media clips inmultiple tracks).

In some embodiments, a media clip or a sequence in the track is playedback by the user selecting a play option (e.g., by clicking on, tapping)displayed in a pop-up menu. In some embodiments, the pop-up menu isaccessed through a cursor control operation (e.g., a right-click cursoroperation or a control-click cursor operation). In some embodiments,playback of the media clip or a sequence is only displayed in theviewing area (i.e., playback is not displayed in the selected media clipor sequence itself).

The playhead 7450 is represented as a T-shape object that spans thetimeline and multiple tracks vertically. It moves horizontally along thecomposite display area (i.e., along the timeline) and its position alongthe timeline indicates the timecode of the composite presentation. Assuch, when the content of a media clip is being played back in theviewing area, the playhead indicates the position of the currentlydisplayed frame in the duration of the composite presentation that themedia clip is a part of. Also, it can indicate the position of thecurrently displayed frame in the duration of the media clip because thebeginning and ending of the duration of the media clip (i.e., the in andout-points of the media clip) is indicated as positions in the timeline.

In the first stage 7435, sequence 7430 is displayed in the compositedisplay area 7410, more specifically in the track 7460. In this example,sequence 7430 is a sequence of media clips 7415, 7420, and 7425 arrangedin that order. There may be other media clips and sequences displayed inthe composite display area but they are not illustrated in this figurefor simplicity. The media clip 7415 has a duration of one minute as itspans from the one hour eight minute mark to the one hour nine minutemark in the timeline 7470. The media clip 7420 has a duration of twominutes. The media clip 7425 has a duration of about one minute andforty seconds as indicated by their span along the time line. In thisstage, the user has just invoked a playback command to start theplayback process. The playhead 7450 is just past the beginning of themedia clip 7415 and a frame of the media clip is displayed in theviewing area 7405.

In the second stage 7440, the playback has reached the one hour tenminute mark of the timeline. This point in the timeline also marks thebeginning of the second minute in the duration of the media clip 7420,which begins at one hour nine minute mark of the timeline 7450. At thispoint, the user invokes a command that causes the in-point of thecurrently playing clip 7420 to be set to the current location of theplayhead. In some embodiments, this edit operation is invoked through ahotkey (e.g., the “a” key). The frame at the beginning of the secondminute in the media clip is displayed in the viewing area 7405.

The third stage 7445 illustrates that the media-editing application hasmodified the in-point of the media clip 7420 such that the new in-pointis the timecode in the source media file for that clip that correspondsto the two minute mark of the previous version of the clip before theedit. As a result, the resulting media clip 7421 now has a duration ofone minute rather than two minutes, as indicated by the span of themedia clip 7420 along the timeline 7420, i.e., the media clip begins atthe one hour and nine minute mark and ends at the one hour and tenminute mark. As the length of the media clip 7420 has been shifted tothe left along the timeline by one minute, the media clip 7425, which isthe next media clip in the sequence, is also shifted by one minute tothe left. The playhead is now located at the beginning of the media clip7420. The frame that is displayed in the viewing area is the first frameof the media clip which is also the frame at the beginning of the secondminute in the previous version of the media clip. The resulting sequenceis sequence 7431.

FIG. 75 illustrates a GUI 7400 of a media-editing application of someembodiments. Specifically, FIG. 75 illustrates extending a media clip7415 by extending its out-point by invoking a command while a sequence7430 of which the media clip 7415 is a part is played back. FIG. 75illustrates the GUI 7400 at four different stages 7535, 7540, 7542, and7544.

In some embodiments, when a media clip in a sequence does not containthe entirety of its source media file, the media-editing applicationallows the user to set a new out-point for the media clip as thesequence is played back. During playback, a playhead moves along themedia clips in the sequence. Before the playhead reaches the end of themedia clip, the user can invoke a command that will cause the media clipto continue playing content from its source file after the currentout-point is reached. When the playhead reaches a location in the mediaclip source at which the user wants to set a new out-point for the mediaclip, the user can invokes a command that will cause the frame at thatlocation to be set as the new out-point for the media clip. As outputpoint of the media clip is extended, a rectangle that represents themedia clip in a track of a composite display area gets extended in someembodiments.

In the first stage 7535, sequence 7430 is displayed in the track 7460 ofthe composite display area 7410. In this example, sequence 7430 is asequence of media clips 7415, 7420, and 7425, arranged in that order.The sequence 7430 is a sequence of media clips 7415, 7420, and 7425arranged in that order. There may be other media clips and sequencesdisplayed in the composite display area but they are not illustrated inthis figure for simplicity. In this stage, the user has just invoked aplayback command to start the playback process. The playhead 7450 isjust past the beginning of the media clip 7415 and a frame of the mediaclip is displayed in the viewing area 7405.

In this example, the source media file for clip 7410 has a duration ofthree minutes, and the current out-point of clip 7415 is set to the endof the first minute of the source media file. The current in-point ofclip 7415 is set to the beginning of the source media file. As such, theduration of the media clip 7415 is one minute as indicated by the spanof the media clip 7415 along the timeline 7450. That is, the media clip7415 spans from on from one hour eight minute mark to one hour nineminute mark in the timeline 7470. The media clip 7420 has a duration oftwo minutes. The media clip 7425 has a duration of about one minute andforty seconds as indicated by their span along the time line.

In the second stage 7540, the playback has reached about the one houreight minutes and fifty second mark of the timeline 7550, which is aboutthe fifty second mark of the first media clip 7515 in the sequence 7530.At this point, the user invokes a command that instructs themedia-editing application to continue playing past the current out-pointof the currently playing clip 7515. In some embodiments, this operationis invoked through a hotkey (e.g., the “d” key). In some embodiments,the playback of clip 7515 continues until the user releases the hotkeyor the media source file ends. If the hotkey is released before thecurrent out-point, then playback continues as it would have if thecommand had not been invoked at all.

In the third stage 7542, the user stops invoking the command byreleasing the hotkey when the playback has past the one hour nine minutemark that marks the end of the media clip 7515 according to its setout-point. They playback has continued past the out-point and theplayhead 7550 is now at the one hour ten minute mark, which isequivalent to the two minute mark of the media clip. Since the user hasreleased the hotkey now, the one hour ten minute mark, or the two minutemark of the media clip is the new out-point of the media clip. Therectangle representing the resulting media clip 7515 has now alsoextended to the one hour ten minute mark, accordingly. The second andthird media clips 7520 and 7525 of the sequence 7530 has been pushed tothe right by one minute as the first media clip 7415 (i.e., the mediaclip 7515 now) has been extended to the right by one minute. Theresulting sequence is sequence 7530.

In the fourth stage 7544, the playback is now at the one hour elevenminute mark of the timeline 7450. Since the user released the hotkey aminute before, thereby marking the end of the first media clip 7415, thesecond media clip 7420 of the sequence has been played back for oneminute as indicated by the current playhead position. However, had theuser been still holding the hotkey, the media clip 7415 would have beenable to extend to this point because the source media file for clip 7415has a duration of three minutes, a minute longer than the media clip7415 after the new out-point was set. Moreover, this point was as far asthe media clip 7415 have been able to be extended because this point isalso the end of the source media file for the media clip.

This Sub-section IV.B has described editing operations on compositesequences. There are more edit operations that some embodiments allowthe user to perform. One of those edit operations is jumping to the nextedit point in the media clips of a sequence while the sequence is beingplayed back. In some embodiments, this edit operation is invoked througha hotkey (e.g., the “'” key). When the hotkey command is invoked whilethe sequence is being played back, the playback jumps to the frame atthe next edit point (e.g., in- or out-points that are later than thecurrent playhead position in the sequence) and the playback continuesfrom that edit point (i.e., from the frame at the next edit point).

Another of those operations is jumping to the previous edit point in themedia clips of a sequence while the sequence is being played back. Insome embodiments, this edit operations is invoked through a hotkey(e.g., the “;” key). When this hotkey command is invoked while thesequence is being played back, the playback jumps back to the frame atthe previous edit point (e.g., in- or out-point that is ahead of thecurrent playhead position in the sequence) and the playback continuesfrom that edit point on (i.e., the frame at the previous edit point).

V. Software Architecture

In some embodiments, the processes described above are implemented assoftware running on a particular machine, such as a computer or ahandheld device, or stored in a computer readable medium. FIG. 76conceptually illustrates the software architecture of a media-editingapplication 7600 of some embodiments. In some embodiments, themedia-editing application is a stand-alone application or is integratedinto another application, while in other embodiments the applicationmight be implemented within an operating system. Furthermore, in someembodiments, the application is provided as part of a server-basedsolution. In some such embodiments, the application is provided via athin client. That is, the application runs on a server while a userinteracts with the application via a separate machine remote from theserver. In other such embodiments, the application is provided via athick client. That is, the application is distributed from the server tothe client machine and runs on the client machine.

The media-editing application 7600 includes a user interface (UI)interaction module 7605, a tagger 7610, a free-form display areacontroller 7615, a set of editing modules 7620, a preview generator7625, a poster frame selector 7630, and a other project data 7635. Themedia production application also includes source files 7640,association manager 7645, and rendering module 7650. In someembodiments, the source files stores the media content (e.g. text,audio, image, and video content) data of media clips. The project datastores data structures for sequences and media clips that include areference to media content data stored as .mov, .avi, .jpg, .png, .mp3,.wav, .txt, etc. files in the source files 7640. In some embodiments,storages 7635 and 7640 are all stored in one physical storage. In otherembodiments, the storages are in separate physical storages, or one ofthe storages is in one physical storage while the other is in adifferent physical storage. For instance, the other project data and thesource files will often be separated.

FIG. 76 also illustrates an operating system 7655 that includes inputdevice driver(s) 7660 and display module 7665. In some embodiments, asillustrated, the device drivers 7660 and display module 7665 are part ofthe operating system 7655 even when the media-editing application is anapplication separate from the operating system.

The input device drivers 7660 may include drivers for translatingsignals from a keyboard, mouse, touchpad, tablet, touch screen, etc. Auser interacts with one or more of these input devices, which sendsignals to their corresponding device driver. The device driver thentranslates the signals into user input data that is provided to the UIinteraction module 7605.

The present application describes a graphical user interface thatprovides users with numerous ways to perform different sets ofoperations and functionalities. In some embodiments, these operationsand functionalities are performed based on different commands that arereceived from users through different input devices (e.g., keyboard,track pad, touchpad, mouse, etc.). For example, the present applicationdescribes the use of a cursor in the graphical user interface to control(e.g., select, move) objects in the graphical user interface. However,in some embodiments, objects in the graphical user interface can also becontrolled or manipulated through other controls, such as touch control.In some embodiments, touch control is implemented through an inputdevice that can detect the presence and location of touch on a displayof the device. An example of such a device is a touch-screen device. Insome embodiments, with touch control, a user can directly manipulateobjects by interacting with the graphical user interface that isdisplayed on the display of the touch-screen device. For instance, auser can select a particular object in the graphical user interface bysimply touching that particular object on the display of thetouch-screen device. As such, when touch control is utilized, a cursormay not even be provided for enabling selection of an object of agraphical user interface in some embodiments. However, when a cursor isprovided in a graphical user interface, touch control can be used tocontrol the cursor in some embodiments.

Moreover, the present application describes the use of hotkeys to invokevarious commands (e.g., editing commands such as trimming, extendingedit points, splitting, marking regions of interest, etc.) to edit amedia clip or a sequence of media clips in some embodiments. In someembodiments, the hotkeys are tangible keys in a typical keyboard (e.g.,keyboard for a desktop computer, keypad for a smartphone, etc.).However, in other embodiments, hotkeys are provided through a touchcontrol. A hotkey in some such embodiments is a key in a graphicalkeyboard rendered on a touch-screen device, any user interface item thatis selectable or clickable, or any other touch-screen methods.

The display module 7665 translates the output of a user interface for adisplay device. That is, the display module 7665 receives signals (e.g.,from the UI interaction module 7605) describing what should be displayedand translates these signals into pixel information that is sent to thedisplay device. The display device may be an LCD, plasma screen, CRTmonitor, touch screen, etc.

The UI interaction module 7605 of media-editing application 7600interprets the user input data received from the input device driversand passes it to various modules, including the tagger 7610, thefree-form display area controller 7615, the editing modules 7620, thepreview generator 7625, and the poster frame selector 7630. The UIinteraction module also manages the display of the UI, and outputs thisdisplay information to the display module 7665. This UI displayinformation may be based on information from the poster frame selector7630, tagger 7610, etc., or directly from input data (e.g., when a usermoves an item in the UI that does not affect any of the other modules ofthe application 7600).

The free-form display area controller 7615 manages the display of andinteraction with the free-form display area of some embodiments. Themodule 7615 generates the free-form display area based on information inthe data structure for the display area and passes this information tothe UI interaction module 7605 for incorporation into the userinterface. The module 7615 also receives user input from the UIinteraction module 7605 to modify the free-form display area (e.g.,moving and piling clips and sequences, adding new clips to the displayarea, concatenating clips into sequences, etc.). The free-form displayarea controller 7615 utilizes the associations manager 7645 in someembodiments to manage the creation and modification of sequences.

The editing modules 7620 receive the various editing commands (trimmingclips and sequences, etc.) and edit the clips and sequences in thefree-form display area accordingly. In addition, editing modules 7620 ofsome embodiments perform edit operations in a composite display area.The editing modules 7620 use the associations manager 7645 formanagement of the sequences when the edits modify the structure of asequence (e.g., a trim that cuts a clip out of a sequence, etc.). Theediting module modifies the clip data structures when clips are edited.

The tagger 7610 receives a clip ID of a tagged media clip, taggedposition of a media clip, and metadata (i.e., tags) from the UIinterface module, and adds this data to the data structure for the mediaclip in some embodiments. In some embodiments, the tagger also creates abin in the project data associated with the metadata. The tagger mayalso perform autosuggestion functions.

The preview generator 7625 enables the output of audio and video fromthe media-editing application so that a user can preview clips orcomposite sequences. The preview generator 7625 uses the project dataand the source media to send display instructions to the UI interactionmodule 7605, which incorporates the information into the user interface.

The poster frame selector 7630 receives input to select a representativeframe of a playing media clip, and identifies the current frame from thepreview generator 7625. The poster frame selector stores thisinformation in the data structure for the playing media clip.

The associations manager 7645 handles the creation and modification ofsequences. When two clips are moved within a particular proximity ofeach other in the free-form display area, the associations manager 7645of some embodiments associates the clips in a sequence. The associationsmanager 7645 creates new data structures for the sequences, and modifiesthe data structures when clips are added to or removed from a sequence,or when clips are moved around within a sequence.

The rendering engine 7650 enables the storage or output of a compositemedia presentation using the media-editing application 7600. Renderingengine 7650 receives data from the editing engine 6820 and/or storages7635 and 7640 and, in some embodiments creates a composite mediapresentation from the source files 7640. The composite mediapresentation can be stored in the storages or a different storage.

While many of the features have been described as being performed by onemodule (e.g., free-form display area controller 7615 or previewgenerator 7625), one of ordinary skill would recognize that thefunctions might be split up into multiple modules, and the performanceof one feature might even require multiple modules.

VI. Process for Defining Media Editing Application

FIG. 77 conceptually illustrates a process 7700 of some embodiments formanufacturing a computer readable medium that stores a media editingapplication such as the application described above by reference to FIG.13. In some embodiments, the computer readable medium is a distributableCD-ROM. As shown, process 7700 begins by defining (at 7705) a free-formdisplay area for displaying media clips. A free-form display area insome embodiments is a composite display area for displaying media clipsover a timeline. For instance, the display areas 700 and 1305 of FIGS. 7and 13 are examples of the defined free-form display area. The process7700 next defines (at 7710) a browser for organizing media clips, suchas the browsers 1310 and 5315 illustrated in FIGS. 13 and 53. Process7700 then defines (at 7715) a composite display area for displayingmedia clips over a time line. For instance, the composite display areas1120 and 1315 of FIGS. 11 and 13 are examples of the defined compositedisplay area.

The process 7700 next defines (at 7720) a viewing area for displayingthe playback of media clips and composite presentations, such as theviewer 1320 and the viewing area 2310 illustrated in FIGS. 13 and 41,respectively. Process 7700 then defines (at 7725) UI items for invokingskimming and playback control commands, such as the playback controls245 and 4355-4365 of FIGS. 2 and 43. Process 7700 also defines (at 7730)edit commands, such as the editing commands described above by referenceto FIGS. 63-66 and 67-73.

Next, the process defines (at 7735) modules for editing media clips andsequences according to user commands, such as the editing modules 7620.The process 7700 then defines (at 7740) a tagger for tagging media clipsand frames according to user commands. The tagger 7610 is one suchtagger. The process 7700 also defines (at 7745) a preview generator forenabling the output of audio and video, such as the preview generator7625. The process 7700 then defines (7750) a module for setting arepresentative frame of a media clip, such as the poster frame selector7630.

Next, the process 7700 defines (at 7755) an association manager formanaging media project data. The associations manager 7645 is an exampleof such defined association manager. Process 7700 then defines (at 7760)a rendering engine for creating a composite media presentation usingsource content according to timeline, such as the rendering module 7650.

The process 7700 then defines (at 7765) other media editing tools andfunctionalities. Examples of such editing tools may include zoom, colorenhancement, blemish removal, audio mixing, trim tools, etc. Inaddition, various other media editing functionalities may be defined.Such functionalities may include library functions, format conversionfunctions, etc. The process 7700 defines these additional tools in orderto create a media editing application that includes other features inaddition to the features described above.

A more detailed view of a media editing application that includes someof these additional features is illustrated in FIG. 78. Specifically,this figure shows a media editing application with these additionaltools. FIG. 78 illustrates a list of video and/or audio clips 7810,video editing tools 7820, and video displays 7830. The list of clips7810 includes video clips along with metadata (e.g., timecodeinformation) about the video clips. In some embodiments, the list ofvideo clips is the list of video clips in a particular sequence of videoclips, and the metadata specifies in and out-points, durations, etc. forthe video clips.

The video editing tools 7820 include tools that allow a user tographically set in and out-points for video clips (in other words, wherein the final product a specific clip or part of a clip will be shown).The video editing tools 7820 can be used to modify the temporal sequenceof the video frame and to synchronize audio tracks with video tracks(e.g., in order to add music over a video clip). In some embodiments,video editing tools 7820 also give users the ability to edit in effectsor perform other video editing functions. In some embodiments, the videoediting tools include trim tools for performing edits such as slideedits, ripple edits, slip edits, roll edits, etc.

Video displays 7830 allow a user to watch multiple video clips at once,thereby enabling easier selection of in and out-points for the videoclips. The screen shot 7800 illustrates a few of many different editingtools that a video editing application of some embodiments may have toedit digital video.

In some cases, some or all of the video clips that are displayed in thelist of clips 7810, played in displays 7830, and edited by a user withvideo editing tools 7820, are video clips of real-world objects (e.g.,people, landscapes, etc.) filmed by a camera and include real-worldaudio (e.g., conversations, real-world noises, etc.) recorded by acamera, microphone, etc. In some cases, some or all of the video clipsare computer-generated animations or include computer generatedanimations (e.g., animated objects, computer-generated effects, etc.).

Returning to FIG. 77, process 7700 next stores (at 7770) the definedmedia-editing application on a non-transitory computer readable storagemedium. As mentioned above, in some embodiments the computer readablestorage medium is a distributable CD-ROM. In some embodiments, themedium is one or more of a solid-state device, a hard disk, a CD-ROM, orother non-volatile computer readable storage medium.

One of ordinary skill in the art will recognize that the variouselements defined by process 7700 are not exhaustive of the modules,rules, processes, and UI items that could be defined and stored on acomputer readable storage medium for a media editing applicationincorporating some embodiments of the invention. In addition, theprocess 7700 is a conceptual process, and the actual implementations mayvary. For example, different embodiments may define the various elementsin a different order, may define several elements in one operation, maydecompose the definition of a single element into multiple operations,etc. In addition, the process 7700 may be implemented as severalsub-processes or combined with other operations within a macro-process.

VII. Computer System

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a non-transitory computer readable storage medium (also referred toas computer readable medium). When these instructions are executed byone or more computational element(s) (such as processors or othercomputational elements like ASICs and FPGAs), they cause thecomputational element(s) to perform the actions indicated in theinstructions. Computer is meant in its broadest sense, and can includeany electronic device with a processor. Examples of computer readablemedia include, but are not limited to, CD-ROMs, flash drives, RAM chips,hard drives, EPROMs, etc. The computer readable media does not includecarrier waves and electronic signals passing wirelessly or over wiredconnections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storagewhich can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs. Finally, any combination ofseparate programs that together implement a software invention describedhere is within the scope of the invention. In some embodiments, thesoftware programs when installed to operate on one or more computersystems define one or more specific machine implementations that executeand perform the operations of the software programs.

FIG. 79 conceptually illustrates a computer system with which someembodiments of the invention are implemented. Such a computer systemincludes various types of computer readable media and interfaces forvarious other types of computer readable media. One of ordinary skill inthe art will also note that the digital video camera of some embodimentsalso includes various types of computer readable media. Computer system7900 includes a bus 7905, processing unit(s) 7910, a graphics processingunit (GPU) 7920, a system memory 7925, a read-only memory 7930, apermanent storage device 7935, input devices 7940, and output devices7945.

The bus 7905 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of thecomputer system 7900. For instance, the bus 7905 communicativelyconnects the processing unit(s) 7910 with the read-only memory 7930, theGPU 7920, the system memory 7925, and the permanent storage device 7935.

From these various memory units, the processing unit(s) 7910 retrieveinstructions to execute and data to process in order to execute theprocesses of the invention. The processing unit(s) may be a singleprocessor or a multi-core processor in different embodiments. While thediscussion in this section primarily refers to software executed by amicroprocessor or multi-core processor, in some embodiments theprocessing unit(s) include a Field Programmable Gate Array (FPGA), anASIC, or various other electronic components for executing instructionsthat are stored on the processor.

Some instructions are passed to and executed by the GPU 7920. The GPU7920 can offload various computations or complement the image processingprovided by the processing unit(s) 7910. In some embodiments, suchfunctionality can be provided using CoreImage's kernel shading language.

The read-only-memory (ROM) 7930 stores static data and instructions thatare needed by the processing unit(s) 7910 and other modules of thecomputer system. The permanent storage device 7935, on the other hand,is a read-and-write memory device. This device is a non-volatile memoryunit that stores instructions and data even when the computer system7900 is off. Some embodiments of the invention use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) asthe permanent storage device 7935.

Other embodiments use a removable storage device (such as a floppy disk,flash drive, or ZIP® disk, and its corresponding disk drive) as thepermanent storage device. Like the permanent storage device 7935, thesystem memory 7925 is a read-and-write memory device. However, unlikestorage device 7935, the system memory is a volatile read-and-writememory, such a random access memory. The system memory stores some ofthe instructions and data that the processor needs at runtime. In someembodiments, the invention's processes are stored in the system memory7925, the permanent storage device 7935, and/or the read-only memory7930. For example, the various memory units include instructions forprocessing multimedia items in accordance with some embodiments. Fromthese various memory units, the processing unit(s) 7910 retrieveinstructions to execute and data to process in order to execute theprocesses of some embodiments.

The bus 7905 also connects to the input and output devices 7940 and7945. The input devices enable the user to communicate information andselect commands to the computer system. The input devices 7940 includealphanumeric keyboards and pointing devices (also called “cursor controldevices”). The output devices 7945 display images generated by thecomputer system. The output devices include printers and displaydevices, such as cathode ray tubes (CRT) or liquid crystal displays(LCD).

Finally, as shown in FIG. 79, bus 7905 also couples computer 7900 to anetwork 7965 through a network adapter (not shown). In this manner, thecomputer can be a part of a network of computers (such as a local areanetwork (“LAN”), a wide area network (“WAN”), or an Intranet, or anetwork of networks, such as the internet. Any or all components ofcomputer system 7900 may be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableBlu-Ray® discs, ultra density optical discs, any other optical ormagnetic media, and floppy disks. The computer-readable media may storea computer program that is executable by at least one processor andincludes sets of instructions for performing various operations.Examples of hardware devices configured to store and execute sets ofinstructions include, but are not limited to application specificintegrated circuits (ASICs), field programmable gate arrays (FPGA),programmable logic devices (PLDs), ROM, and RAM devices. Examples ofcomputer programs or computer code include machine code, such as isproduced by a compiler, and files including higher-level code that areexecuted by a computer, an electronic component, or a microprocessorusing an interpreter.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium” and “computer readable media” are entirelyrestricted to tangible, physical objects that store information in aform that is readable by a computer. These terms exclude any wirelesssignals, wired download signals, and any other ephemeral signals.

Although the above description of the various embodiments of theinvention show video editing, many examples of this free form space canbe used for audio editing as well. While the invention has beendescribed with reference to numerous specific details, one of ordinaryskill in the art will recognize that the invention can be embodied inother specific forms without departing from the spirit of the invention.In addition, a number of the figures (including FIGS. 21, 30, 47, 50,55, 58, and 67) conceptually illustrate processes. The specificoperations of these processes may not be performed in the exact ordershown and described. The specific operations may not be performed in onecontinuous series of operations, and different specific operations maybe performed in different embodiments. Furthermore, the process could beimplemented using several sub-processes, or as part of a larger macroprocess. Thus, one of ordinary skill in the art would understand thatthe invention is not to be limited by the foregoing illustrativedetails, but rather is to be defined by the appended claims.

We claim:
 1. A non-transitory computer-readable medium storing amedia-editing application for creating media presentations, saidapplication comprising a graphical user interface (GUI), said GUIcomprising: a display area for (i) organizing media clips and (ii)compositing media clips by creating an association between a set of themedia clips to form a concatenated sequence of media clips that are tobe played back sequentially; a timeline for organizing and compositing amedia presentation; a user interface tool for adding to the timeline theconcatenated sequence of media clips from the display area; and aviewing area for playing back a media clip.
 2. The non-transitorycomputer-readable medium of claim 1, wherein said GUI further comprisesa media library area for selecting media clips to be added to thedisplay area.
 3. The non-transitory computer-readable medium of claim 1,wherein each media clip has a first graphical representation in thedisplay area and has a second graphical representation, different thanthe first graphical representation, in the timeline.
 4. Thenon-transitory computer-readable medium of claim 1, wherein each of aplurality of media clips is a video clip, wherein a video clip has athumbnail representation in the display area, wherein the video clip'scontent is playable in the video clip's thumbnail representation.
 5. Thenon-transitory computer-readable medium of claim 1, wherein the timelineis further for compositing sequences of media clips added from thedisplay area to produce the media presentation.
 6. The non-transitorycomputer readable medium of claim 4, wherein the viewing area is furtherfor concurrently playing the content of the video clip as the content isalso being played back in the video clip's thumbnail representation. 7.The non-transitory computer-readable medium of claim 1, wherein mediaclips in the display area are attachable to each other in order to formthe concatenated sequence of media clips.
 8. The non-transitorycomputer-readable medium of claim 7, wherein within the display area,each media clip has a graphical representation, a portion of which isdefined to attract and attach to portions of the graphicalrepresentations of other media clips.
 9. The non-transitorycomputer-readable medium of claim 1, wherein within the display area,each media clip has a graphical representation, wherein the GUI furthercomprises a set of tools for marking with a visual marker a portion of afirst media clip while the first media clip is being played back,wherein the marked portion of the first media clip represents a portionof media content associated with the first media clip.
 10. Thenon-transitory computer-readable medium of claim 9, wherein the GUIspecifies the visual marker as a selectable user interface (UI) itemthat allows the marked portion of the first media clip to be extractedfrom the first media clip to create a second media clip, wherein thesecond media clip represents media content that is the portion of themedia content associated with the first media clip.
 11. Thenon-transitory computer-readable medium of claim 1, wherein said displayarea does not include any smaller display areas within it.
 12. A methodof defining a media-editing application for creating mediapresentations, said application comprising a graphical user interface(GUI), the method comprising: in the GUI, defining a display area for(i) organizing media clips and (ii) compositing media clips by creatingan association between a set of the media clips to form a concatenatedsequence of media clips that are to be played back sequentially;defining a timeline for (i) organizing and compositing a mediapresentation and (ii) receiving the concatenated sequence of media clipsfrom the display area for placement within the media presentation; anddefining a viewing area for playing back a media clip.
 13. The method ofclaim 12, wherein the display area is a first display area, the methodfurther comprising: in the GUI, defining a second display area fororganizing and compositing media clips, and defining a set of operationsfor moving media clips between the first display area, the seconddisplay area, and the timeline.
 14. The method of claim 12, wherein eachof a plurality of media clips represents a piece of video content,wherein each video content piece includes a plurality of frames, whereineach media clip includes a graphical display that displays a thumbnailimage of one of the frames of the media clip's associated video content,the method further comprising defining a set of tools for allowing auser to change the graphical display of a media clip from a thumbnailimage of one frame of the media clip's associated video content toanother thumbnail image of another frame of the media clip's associatedvideo content.
 15. The method of claim 12, wherein each media clip has afirst graphical representation in the display area and has a secondgraphical representation, different than the first graphicalrepresentation, in the timeline.
 16. The method of claim 12 furthercomprising defining a graphical representation for each media clip inthe display area to include a set of locations that attract and attachto a set of locations of the graphical representations of other mediaclips.
 17. The method of claim 12, wherein within the display area, eachmedia clip has a graphical representation, wherein the method furthercomprises: defining a set of tools for marking with a selectable visualmarker a portion of a first media clip while the first media clip isbeing played back, wherein the marked portion of the first media cliprepresents a portion of media content associated with the first mediaclip; defining a set of operations for allowing the marked portion ofthe first media clip to be extracted from the first media clip to createa second media clip, wherein the second media clip has media contentthat is the portion of the media content associated with the first mediaclip.
 18. A non-transitory computer-readable medium storing amedia-editing application for creating media presentations, saidapplication comprising sets of instructions for: displaying a firstdisplay area for compositing sets of media clips to form a concatenatedsequence of media clips, the media clips of each sequence of media clipsto be played back sequentially; and displaying a second display area fororganizing and compositing media clips for placement within a mediapresentation; providing a user interface tool for adding to the seconddisplay area the concatenated sequence of media clips from the firstdisplay area; and displaying a viewing area for playing back thesequence of media clips.
 19. The non-transitory computer-readable mediumof claim 18, wherein the application further comprises a set ofinstructions for displaying a media library area for selecting mediaclips to be added to the first display area.
 20. The non-transitorycomputer-readable medium of claim 18, wherein each media clip has afirst graphical representation in the first display area and has asecond graphical representation, different than the first graphicalrepresentation, in the second display area.
 21. The non-transitorycomputer-readable medium of claim 18, wherein the application furthercomprises a set of instructions for displaying each media clip in thefirst display area as a graphical representation that has a set oflocations that attract and attach to a set of locations of the graphicalrepresentations of other media clips.
 22. The non-transitorycomputer-readable medium of claim 18, wherein within the first displayarea, each media clip has a graphical representation, wherein theapplication further comprises sets of instructions for: while a firstmedia clip is being played back, receiving instructions to mark with aselectable visual marker a portion of the first media clip, wherein themarked portion of the first media clip represents a portion of mediacontent associated with the first media clip; receiving selection andmovement of the visual marker of the first media clip; in response tothe selection and movement of the visual marker, extracting the markedportion from the first media clip to create a second media clip, whereinthe second media clip represents media content that is the portion ofthe media content associated with the first media clip.
 23. Thenon-transitory computer-readable medium of claim 18, wherein the firstdisplay area allows more than one set of media clips to be concatenatedto form more than one sequence of the media clips that are concurrentlydisplayed in the first display area; wherein the media clips within eachsequence have a temporal relationship to one another, but two differentsequences that do not abut in the first display area do not have thetemporal relationship with one another.